Re: [Tofoo] VXLAN (UDP tunnel protocols) and non-zero checksums

"Larry Kreeger (kreeger)" <> Thu, 01 May 2014 20:11 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 6EB361A6FD1; Thu, 1 May 2014 13:11:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -10.151
X-Spam-Status: No, score=-10.151 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RP_MATCHES_RCVD=-0.651, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id QP4HMWMv7ilD; Thu, 1 May 2014 13:11:02 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 4B2821A700D; Thu, 1 May 2014 13:11:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;;; l=21395; q=dns/txt; s=iport; t=1398975060; x=1400184660; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=f36ylRAJYDtaqG5Eqa170OZdFxz87/Y+kD0CZaJBVL0=; b=AWRWbLq5pomXscU45qZaQbAlna8LQA1/iEcL9WVBQAGitMZghszCRkFf VHH0CQiGpLtf2ZTGIbwh5OYXBBMcuQk27JzqjBa62XQGk3YDJ5uknqqe2 Q+E6eNenFHV9aXPArNUifUqGM0KvWT7q+9kHVx+A+9NhL5tyGX0I+wCUr o=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos; i="4.97,966,1389744000"; d="scan'208,217"; a="40358871"
Received: from ([]) by with ESMTP; 01 May 2014 20:10:59 +0000
Received: from ( []) by (8.14.5/8.14.5) with ESMTP id s41KAxv1030187 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Thu, 1 May 2014 20:10:59 GMT
Received: from ([]) by ([]) with mapi id 14.03.0123.003; Thu, 1 May 2014 15:10:59 -0500
From: "Larry Kreeger (kreeger)" <>
To: Tom Herbert <>
Thread-Topic: [Tofoo] VXLAN (UDP tunnel protocols) and non-zero checksums
Thread-Index: AQHPZKbjNYPI8b/6TUi7fhWXO3OBYJsquXGAgACP5oD//4xngIAAi68AgACmqQA=
Date: Thu, 1 May 2014 20:10:58 +0000
Message-ID: <>
References: <> <> <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/
x-originating-ip: []
Content-Type: multipart/alternative; boundary="_000_CF87F70EF437Ckreegerciscocom_"
MIME-Version: 1.0
Cc: "" <>, "" <>, "" <>, "" <>
Subject: Re: [Tofoo] VXLAN (UDP tunnel protocols) and non-zero checksums
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Discussion list for Tunneling over Foo \(with\)in IP networks \(TOFOO\)." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 01 May 2014 20:11:12 -0000

From: Tom Herbert <<>>
Date: Wednesday, April 30, 2014 8:14 PM
To: Larry Kreeger <<>>
Cc: "<>" <<>>, "<>" <<>>, "<>" <<>>, Dinesh Dutt <<>>
Subject: Re: [Tofoo] VXLAN (UDP tunnel protocols) and non-zero checksums

On Wed, Apr 30, 2014 at 6:54 PM, Larry Kreeger (kreeger) <<>> wrote:
See inline, marked with LK>. - Larry

From: Tom Herbert <<>>
Date: Wednesday, April 30, 2014 6:48 PM
To: Larry Kreeger <<>>
Cc: "<>" <<>>, "<>" <<>>, "<>" <<>>, Dinesh Dutt <<>>
Subject: Re: [Tofoo] VXLAN (UDP tunnel protocols) and non-zero checksums

On Wed, Apr 30, 2014 at 5:13 PM, Larry Kreeger (kreeger) <<>> wrote:
Hi Tom,

I'll give you my perspective on why I feel the behavior described in the
VXLAN draft is a good thing.  First, it is my assumption that some
implementations (e.g. many hardware implementations), will not implement
checksum generation, nor checksum validation.  I believe this is an
implementation reality.  If implementations are required to check a
non-zero checksum, but can't actually do it, what alternatives do they

Barring that the implementations you're referring to only implement IPv6, these implementations must be already be doing checksum validation for IPv4 header-- so the checksum logic must have been implemented and I'm not sure the argument that this HW can't calculate the UDP csum would have much merit. A specific example implementation would be nice here, for instance in the case that the stack decapsulates the checksum can always be done in SW if verification is not provided by the device.

LK> I am referring to switching ASIC implementations.  There is no software nor traditional UDP stack involved.

Are these implementations also ignoring IPv4 header checksum then?

LK2> Yep.  The text in the VXLAN draft applies to both IPv4 and IPv6.  In fact, IPv6 was only added to the draft in a later version and all original implementations supported only IPv4 encapsulation (I'm not aware of any IPv6 implementations, but maybe there are some out there).

Also, as I pointed out, the UDP checksum is *not* useless in VXLAN. In an L3 routed network there is nothing that protects the vni from corruption expect possibly the UDP checksum. Without any additional security mechanisms is VXLAN header (like a cookie), the only way I could deploy VXLAN is with checksums enabled. So in my opinion, the draft should be encouraging use of UDP checksums instead of discouraging them.

LK> Neither I nor the VXLAN draft claims that UDP checksums are useless, however I think it is an exaggeration to say "nothing" protects the VNI from corruption when the packet is being carried over Ethernet with a robust CRC32 protecting it.

Assuming we are using Ethernet (I don't believe this can be a requirement either) this only provides hop to hop protection, not end to end. I don't have a completely error free network and checksum errors while low, are non-zero.

LK2> I suppose it depends on the routers and switches in use, but a well designed one should not flip any payload bits while routing/switching (e.g. use internal CRCs across its backplane).

Sending a packet to the wrong VM is fundamentally bad and could be quite costly,  so for me UDP checksums would be the minimum requirement. This would need to apply to the case where the protocol is switched also.

LK2> It is much easier for software on hosts to do this (and maybe NICs).  You might be hard pressed to find a hardware gateway that does it in a cost effective manner.

In any case, the requirements and constraints for when it's acceptable to ignore non-zero checksums for UDP should be specified in a more general way if is being allowed.  Maybe an addendum to RFC 6935?



One is to drop all packets with a non-zero checksum (because one
might be invalid and even one invalid one slipping through would be
unacceptable).  Another alternative is to accept all checksum values.  The
second option greatly enhances interoperability with implementations that
choose to generate a checksum and implementations that cannot validate the
checksum.  It allows a mixed environment where "better" implementations
(that can validate) can interoperate with "inferior" implementations that
are unable to validate the checksum.

BTW, VXLAN is not the only tunneling protocol to specify this behavior.
LISP (RFC 6830), specifies exactly the same behavior.
 - Larry

On 4/30/14 12:01 PM, "Tom Herbert" <<>> wrote:

>I noticed that the VXLAN draft allows an implementation to potentially
>ignore a non-zero invalid UDP checksum.
>"When a decapsulating endpoint receives a packet with a non-zero
>checksum it MAY choose to verify the checksum"
>However, from RFC 1122:
>"If a UDP datagram is received with a checksum that is non-zero and
>invalid, UDP MUST silently discard the datagram."
>It doesn't seem like 1122 allows checksum verification to be optional,
>so these would seem to be a conflict. Presumably, ignoring the RX csum
>is included for performance but since the sender can already send zero
>checksums in UDP (they are optional in IPv4 and allowed for IPv6
>tunnels in RFC 6935) I'm not sure this is necessary. Besides that, the
>UDP checksum is potentially the only thing that protection of the vni
>against corruption end to end so allowing a receiver to ignore a bad
>checksum seems very risky.
>As a comparison, RFC 3931 (L2TP) has the following wording:
>"Thus, UDP checksums MAY be disabled in order to reduce the associated
>packet processing burden at the L2TP endpoints."
>This is somewhat ambiguous, but seems like the correct interpretation
>should be that zero checksums may be sent with L2TP/UDP, but on
>receive non-zero checksums should still be validated.
>Are these interpretations correct? Is there there a need to clarify
>the requirement for UDP tunnel protocols and checksums?
>Tofoo mailing list