Re: [nvo3-dt-encap] [nvo3] Encap draft published by design team

Tom Herbert <> Thu, 09 February 2017 20:05 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id CE105129586 for <>; Thu, 9 Feb 2017 12:05:44 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id bZDDWSEugNAM for <>; Thu, 9 Feb 2017 12:05:43 -0800 (PST)
Received: from ( [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id E080F12946E for <>; Thu, 9 Feb 2017 12:05:42 -0800 (PST)
Received: by with SMTP id k15so14975577qtg.3 for <>; Thu, 09 Feb 2017 12:05:42 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=CZCHSbc/Nu6+gs8n7vfRbz5+S6PJtmMgAt5jMx5KCDQ=; b=g0l3imbPwmltafjnwcHAUkWbKcfHltuVY0307oRVkbTPe/WvOY1aA7aioD7LxlFTd1 l7QA6QbHVHzrSm2cW+wyY7kcLCd54ztWOKZMyJCyRa2M6NFJOMgPHEBbpniPIAqulCN5 bvQHXzzUKyB5asSotaP4NrHJJejUQKiQHZb3HpZUhvW7cEI+aYRrkmxjAkh8Dz53W2+5 iX0bs1KiXHjraPV60ahqp8CUy/2k+vO9x20+wWyXJnDXU5VhNNLq1sCBnodN29pYiRlV Q55JpZzOI/3oV/1Bt1peo+IZs/bs42izwxkKr6jZ81Mx3aIFAedlHsNQVHPnnXQWUtv6 0lCA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=CZCHSbc/Nu6+gs8n7vfRbz5+S6PJtmMgAt5jMx5KCDQ=; b=k/UyaH6jAn4PWXSJ1KDZNtP0LC24NzTWwh5Tu973tSU5K5zefskXcM/tBHjFz6WK0c TLXB5hz2DTgzCX2wgumVP6NbMzEUNfR03aM6nJ4fjSZILk2ntP7XiikMb0TMd6dpObWY jTwuKkCXybxO42NXpRoG2YF0rjrS6b5z+lAoHGTSyohiMyNwj5kE0BqzUMzUdFAG2E3L crqrhu0lcET1SqaZYQaAHsexC49I1SJwgBidurwpd6zi4JytjizFujVsZux9Eajq10oX Nq2oeaAfmr73tJ0AtooTg7rZVXpL0qcw9aphrNyX7glLyPqWdOY/o4dsa/FBT7jA4ZAa 4k9Q==
X-Gm-Message-State: AMke39nm/XhcPF6OJWkbriwpYW9GO/tN3VaLREEk2WJ8C5kIl7IUrgmW6I2RgRmRHDy4fzJVfCbH9j1SfXcGAg==
X-Received: by with SMTP id i20mr5107120qtc.247.1486670741902; Thu, 09 Feb 2017 12:05:41 -0800 (PST)
MIME-Version: 1.0
Received: by with HTTP; Thu, 9 Feb 2017 12:05:41 -0800 (PST)
In-Reply-To: <>
References: <> <> <> <>
From: Tom Herbert <>
Date: Thu, 9 Feb 2017 12:05:41 -0800
Message-ID: <>
To: Joe Touch <>
Content-Type: text/plain; charset=UTF-8
Archived-At: <>
Cc: Sam Aldrin <>, "" <>,,
Subject: Re: [nvo3-dt-encap] [nvo3] Encap draft published by design team
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Private mailing list for internal NVO3 Encapsulation Design Team discussions <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 09 Feb 2017 20:05:45 -0000

On Thu, Feb 9, 2017 at 11:42 AM, Joe Touch <> wrote:
> Hi, Tom,
> On 2/9/2017 11:33 AM, Tom Herbert wrote:
>> On Thu, Feb 9, 2017 at 10:46 AM, Joe Touch <> wrote:
>>> ...
>>> -  the discussion of TLV vs. bitfields needs a bit more expansion. E.g., TLV
>>> necessarily requires serial processing, whereas bitfields can be processed
>>> in parallel.
>> It looks like the intent is to have a control plane that defines the
>> TLVs that will be used, a required ordering, and the assumption that
>> all TLVs are all the same size. If that is true then parsing TLVs can
>> be turned into a parallel problem, e.g. that can be accomplished with
>> 2^N entries in a TCAM the same as bit fields.
> That's true only if the TLVs in use are known in advance - if that's
> true, then the set of TLVs effectively becomes a large, static bitfield
> in the context of that tunnel.
Yep, in that case even the meanings of the bit fields would haven't to
fixed the whole data packet format could just be negotiated by the
control plane. The only issue with that is if intermediate boxes are
expected to parse the optional data then they need to be privy to what
was negotiated, but that would also be true if we expected middleboxes
to be able to efficiently parse TLVs and they need to see the control
plane negotiation to set up the processing path.

> If that's true, then easy hardware processing is possible only in the
> context of each individual tunnel, which requires a reconfigurable
> hardware parser for each tunnel.
> Otherwise, the only way to know where the Nth TLV field starts is to
> parse through the N-1th, which - recursively - reduces to requiring
> iterative processing.
Yes, and iterative processing of an open ended list of TLVs is still
hard to do in hardware _and_ software and in fact is an obvious DOS
vector. Last I checked OpenFlow, P4, and eBPF don't fully implement
loops that are needed to process an arbitrary list of TLVs (they
resolved this through some hackery and artificial constraints). The
unlimited flexibility of TLVs does not come without a cost...


>> But, now implies that
>> explicit tunnels need to be established by the control plane before
>> the data plane can be used, and to be practical probably means that a
>> node will expect to receive pretty much the same set of TLVs with the
>> same order from all communicating devices (or maybe the intent is to
>> have a universal ordering of TLVs?).
> (agreed - the "known in advance case, as above)
>> It would be good to have a reference to other low layer protocols that
>> include TLVs or the like, especially IPv4 and IPv6, and an explanation
>> as to how TLVs in nvo3 will address all the issues of those protocols
>> that have deterred deployment of TLVs.
> Agreed, especially if those are the "inner" headers that might need to
> be parsed anyway.
> Joe