Re: [Rtg-dt-encap-considerations] [nvo3] Alignment and Ethernet encapsulation

Tom Herbert <> Tue, 20 September 2016 18:38 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id ECAED12B144 for <>; Tue, 20 Sep 2016 11:38:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id xWiJ7dXEyLw5 for <>; Tue, 20 Sep 2016 11:38:41 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:400d:c0d::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 780D9128B38 for <>; Tue, 20 Sep 2016 11:38:41 -0700 (PDT)
Received: by with SMTP id l91so11781319qte.3 for <>; Tue, 20 Sep 2016 11:38:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=r9MxcyFmjIzO73hxS0mWen4Na2pt4NogL4Z7pYL7slg=; b=Xcb32IeW+VJnqJoKeDrimcsaUbLTL60efuaJwM/kWmQumxOPm1eg9iLxIawRKsyEPK HFz36s75yQgWaEtvl5h8LQuO/eBDZ1nqeQVUNuIaqxV+bVt2WvSqr8rR35JTCrPwTUkH l/V61creC/mE/3o+e+SxMgvD3aWlNK772QbZ856gMViQ5cP8ZWckM8r/W7EfZiFYYN9J cffBH9mVH+SYn0wao+qxx6x46axfAKZhARpVbGVvj4BqdvVKUOykvoAbp0873ooeOd+C y2TdMCFoQb5tRtHrspL8YoNcvSbOORLRdCxGIKOK4dxQMGIbugL1oPtyBOsaMl+UqAhc eXxA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=r9MxcyFmjIzO73hxS0mWen4Na2pt4NogL4Z7pYL7slg=; b=W+eM0bbmiF+eCBA+tHoHla0uHB9HVjf5Yd/OpN2QeuOWanHaoD9wvatYsdEb5e2ZkL XuFuqw6atmNE46GA6TU7mddDnvXXp6y3Nlh0pRHs5rZtFRKTtqOjI7EEoVkwhZ6VuQ1Q ADCFqeoqozrmea79TObk+1xQit9l+0Xm8/6WeOnUBSw7AV8pH/7jx4Os1JFDQtXQok+q O6Qd5zApyqBbPuPeGK5ZJ8onM3Lq6gazjXSUwhgI2zr1obbxEnoNR4HFMiu5OW14O0QL dRiuwMaIX5yfHibVriB1kKfu8U/RGbalLH9A6EAx+hrQYQ9LLFGOan0FUqhc2yK0nCPQ CD9w==
X-Gm-Message-State: AE9vXwMsqGLOgAQSPzzQBUma2Y23dDbD9I49f9SMf94i61DDoIu22GkD7hmini9iHyDj578FQLdJ5yPntOuMeA==
X-Received: by with SMTP id r8mr37356922qtb.120.1474396720530; Tue, 20 Sep 2016 11:38:40 -0700 (PDT)
MIME-Version: 1.0
Received: by with HTTP; Tue, 20 Sep 2016 11:38:40 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <> <> <>
From: Tom Herbert <>
Date: Tue, 20 Sep 2016 11:38:40 -0700
Message-ID: <>
To: Joe Touch <>
Content-Type: text/plain; charset=UTF-8
Archived-At: <>
Cc: Sowmini Varadhan <>, "" <>, "" <>
Subject: Re: [Rtg-dt-encap-considerations] [nvo3] Alignment and Ethernet encapsulation
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Routing Area Design Team on Encapsulation Considerations discussion list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 20 Sep 2016 18:38:43 -0000

On Tue, Sep 20, 2016 at 10:55 AM, Joe Touch <> wrote:
> On 9/20/2016 10:47 AM, Tom Herbert wrote:
>> On Tue, Sep 20, 2016 at 10:35 AM, Joe Touch <> wrote:
>>> ...
>>> If you've been handed Ethernet to encapsulate, then that is the header
>>> whose alignment you should be optimizing.
>>> As you point out, you can't align both IP and Ethernet to 4-byte
>>> boundaries at the same time.
>> You can if you insert a two byte pad before the encapsulated Ethernet
>> header like ETHERIP (RFC3378 does).
> You CAN do a lot of things, but you shouldn't.
> If you're handed ethernet to encapsulate, then THAT is the header whose
> access should be aligned.
> You have no business ASSUMING that you need to optimize for IP header
> access. All you're doing is messing up the ethernet access.
Sorry, but I disagree with that statement. Keeping IP headers aligned
for stack processing is a HUGE concern in real implementation.
Probably every device driver on the planet will put in a two byte pad
in receiver buffers to ensure that the Ethernet payload (and hence IP
header, EHs, TCP headers, etc.). We need two byte alignment of
Ethernet headers, and four byte alignment on IP protocols. If we don't
have this a lot of things will break (as VXLAN is currently broken on
SPARC)! Maybe in theory alignment is no longer considered an issue and
is just a historical footnote, but on real devices in deployment today
unaligned protocol headers still can be problematic.


> Frankly, anything that already parses Ethernet then IP ought to be able
> to efficiently handle the difference in the two alignments - and you
> might be messing THAT up.
> RFC3378 defines a header used for ETHERIP; that header may or may not be
> relevant to other ethernet-in-X encapsulations. The field needed to be
> there for versioning and was expanded from 8 to 16 bits, which certainly
> helps align the *ethernet* header that comes after it compared to the
> earlier use of an 8-bit field. There's no mention of aligning for the
> purposes of accessing the IP header, which again, I disagree with.
> Joe