Fwd: Re: RtgDir review: draft-rtg-dt-encap-02

Erik Nordmark <nordmark@acm.org> Mon, 06 July 2015 17:33 UTC

Return-Path: <nordmark@acm.org>
X-Original-To: rtgwg@ietfa.amsl.com
Delivered-To: rtgwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com []) by ietfa.amsl.com (Postfix) with ESMTP id 695DB1A8A3C for <rtgwg@ietfa.amsl.com>; Mon, 6 Jul 2015 10:33:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.935
X-Spam-Status: No, score=-1.935 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_SOFTFAIL=0.665] autolearn=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id rEqjKGp_cetq for <rtgwg@ietfa.amsl.com>; Mon, 6 Jul 2015 10:33:46 -0700 (PDT)
Received: from d.mail.sonic.net (d.mail.sonic.net []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E88681A6F12 for <rtgwg@ietf.org>; Mon, 6 Jul 2015 10:33:45 -0700 (PDT)
Received: from [] ([]) (authenticated bits=0) by d.mail.sonic.net (8.15.1/8.15.1) with ESMTPSA id t66HXZ15012270 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 6 Jul 2015 10:33:37 -0700
Subject: Fwd: Re: RtgDir review: draft-rtg-dt-encap-02
References: <559AB820.9000309@acm.org>
To: "rtgwg@ietf.org" <rtgwg@ietf.org>
From: Erik Nordmark <nordmark@acm.org>
X-Forwarded-Message-Id: <559AB820.9000309@acm.org>
Message-ID: <559ABBEF.2020803@acm.org>
Date: Mon, 6 Jul 2015 19:33:35 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.0.1
MIME-Version: 1.0
In-Reply-To: <559AB820.9000309@acm.org>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Sonic-CAuth: UmFuZG9tSVap+vxuvZbaTvnbJ9siFN5DQdfdRSIaOXidY8PVHrVBIdBMrLqVc0mZXyb8ndaReNvhlHfMIvpm/b8jqvnfne+i
X-Sonic-Spam-Details: 0.0/5.0 by cerberusd
Archived-At: <http://mailarchive.ietf.org/arch/msg/rtgwg/tnFRtHLvUcE4jY1s6aFL4niq544>
X-BeenThere: rtgwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Routing Area Working Group <rtgwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rtgwg>, <mailto:rtgwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rtgwg/>
List-Post: <mailto:rtgwg@ietf.org>
List-Help: <mailto:rtgwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rtgwg>, <mailto:rtgwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Jul 2015 17:33:49 -0000

-------- Forwarded Message --------
Subject: 	Re: RtgDir review: draft-rtg-dt-encap-02
Resent-Date: 	Mon, 6 Jul 2015 10:19:18 -0700 (PDT)
Resent-From: 	nordmark@acm.org
Resent-To: 	draft-rtg-dt-encap@ietf.org, rtg-ads@ietf.org
Date: 	Mon, 6 Jul 2015 19:17:20 +0200
From: 	Erik Nordmark <nordmark@acm.org>
To: 	Jamal Hadi Salim <hadi@mojatatu.com>om>, rtg-ads@tools.ietf.org 
<rtg-ads@tools.ietf.org>rg>, rtgwg@ietf.org <rtgwg@ietf.org>
CC: 	rtg-dir@ietf.org <rtg-dir@ietf.org>rg>, draft-rtg-dt-encap@tools.ietf.org

On 6/30/15 2:20 PM, Jamal Hadi Salim wrote:
> I have been selected as the Routing Directorate reviewer for this draft.
> The Routing Directorate seeks to review all routing or routing-related
> drafts as they pass through IETF last call and IESG review, and sometimes
> on special request. The purpose of the review is to provide assistance to
> the Routing ADs. For more information about the Routing Directorate,
> please see http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir
> Although these comments are primarily for the use of the Routing ADs, it
> would be helpful if you could consider them along with any other IETF
> Last Call comments that you receive, and strive to resolve
> them through discussion or by updating the draft.
> Document: draft-rtg-dt-encap-02
> Reviewer: Jamal Hadi Salim
> Review Date: 6/30/15 (later than requested, sorry)
> Intended Status: Informational
> WG LC End Date: unknown
> Summary:
> The document has significant good work and recommendations for
> encapsulation design. Many years of experience in issues found
> with encapsulation deployments are discussed. There are times
> where i lost track what the document was about because issues
> were being discussed without making recommendations on what is needed
> from an encapsulation perspective to deal with those issues; otoh,
> a good read is section 18 which would mention an issue and in the
> same breath suggests how a design should handle said issue.

Thanks for your careful review.

The design team tried to stay away from potentially contentions
discussion by putting firm recommendations in place. Thus currently the
document is more of "things to think about" and some tradeoffs, and less
of recommendations (and no requirements).

Alia has asked for more of a list of choices for the protocol designers,
and this is definitely something which folks can help with as we
continue this work in the RTGWG. But I also think we need to let the
specific in-flight WGs (NVO3, SFC, and BIER) use this document as input
but figure out their own tradeoffs and answers.
> The document needs at least one more pass.
Agreed. We're looking for more feedback in the RTGWG and also from the
WGs that work on the encapsulations.

> I have some minor concerns about this document that I believe are
> resolvable.
> Annotated comments attached.
>     o  SFC carries service meta-data which might be modified or
>        unmodified as the packets follow the service path.  SFC talks of
> Being a little picky, how about:
> "SFC carries service meta-data which might be modified as the packets
> follow the service path."
Agreed - the "unmodified" is a bit to clever (intended to capture that
in some cases things might be restored to their original).

> 6.  Terminology
>     The capitalized keyword MUST is used as defined in
>     http://en.wikipedia.org/wiki/Julmust
> Missing the context on what looks like a high calorie delicious drink.
> and should that be https?;->

Since you are the first to discover the lack of https in that URL, you
get a free beverage of your choice (in the bar in Prague if you will be

>     The UDP source port might change over the lifetime of an encapsulated
>     flow, for instance for DoS mitigation or re-balancing load across
>     ECMP.
> Shouldnt the above statement bear a little more discussion/comment?
> What happens to packet ordering then?
I'll add a flag about this.

>        [Note: For any given bit in BIER (that identifies an exit from the
>        BIER domain) there might be multiple immediate next hops.  The
>        BIER entropy field is used to select that next hop as part of BIER
>        processing.  The BIER forwarding process may do equal cost load
>        balancing, but the load balancing procedure MUST choose the same
>        path for any two packets have the same entropy value.]
> "... two packets that have the same ..."

>     o  In the case of IP transport use >=14 bits of UDP source port, plus
>        outer IPv6 flowid for entropy.
> Looks like a typo.  <=14 bits?
No; 14 or 16 depending on the environment. I'll reword as such.

>     o  Reuse Ethernet types - makes it easy to carry existing L2 and L3
>        protocols including IPv6, IPv6, and Ethernet.  Disadvantages are
>        that it is a 16 bit number and we probably need far less than 100
>        values, and the number space is controlled by the IEEE 802 RAC
>        with its own allocation policies.
> If i understood correctly what "reuse" implies: you are suggesting a new
> super-ethertype whose content space will carry an additional type
> semantic so you never have to go back to IEEE?
Nope. Rewording as "Use the Ethernet type space" should make it clearer
- means going to the IEEE RAC for each new protocol. (Not my idea - IDs
have suggested using the Ethernet type space.)

I'll apply s/Reuse/Use/ to the next bullets as well.

>     Many Internet protocols use fixed values (typically managed by the
>     IANA function) for their next-protocol field.  That facilitates
>     interpretation of packets by middleboxes and e.g., for debugging
>     purposes, but might make the protocol evolution inflexible.  Our
>     collective experience with MPLS shows an alternative where the label
>     can be viewed as an index to a table containing processing
>     instructions and the table content can be managed in different ways.
> Would it not be useful to provide a reference here? Just reading this
> has questions popping for me - who populates this tag-indexed table of
> instructions and could interop be impacted?
The DT added the above text based on comments at the mike from Stewart
Bryant. I don't know if there is any reference for the general concept.
Anybody else know?

In general this implies some control-plane mechanism to populate the table.

>     o  Would it be useful for the IETF come up with a common scheme for
>        encapsulation protocols?  If not each encapsulation can define its
>        own scheme.
> In my view it would be hard to come up with a ring to rule them all.
> There are cases where simple is good enough and asking someone to carry
> a christmas tree is the wrong answer. And, yes, there are cases where
> (to quote Mencken) the answer is clear, simple and wrong (especially
> in one-off-use-cases which then are refactored to fit into square pegs).
> My suggestion is to not be too clever in answering the question above.
FWIW the design team just asked the question; someone else gets to
answer ;-)

But I agree to not blindly forcing some new requirement on common
next-header approaches across different WGs.

> Reference to Aerolink and the sins committed would be useful.
> I googled aerolink and found references of some radio thing running over IP.
> Given IP provides the fragmention service above, why is aerolink not capable
> of this mechanism? I think there's a simple answer; just reading this didnt help.
There are Aerolink internet-drafts. We'll have to go back and see what
makes sense as citations.

>     o  Make OAM packets look the same as data packets i.e. the initial
>        part of the OAM payload has the inner Ethernet, IP, TCP/UDP
>        headers as a payload.  (This approach was taken in TRILL out of
>        necessity since there is no UDP header.)  Any OAM bit in the
>        encapsulation header must in any case be excluded from the
>        entropy.
> Does it make sense to have inband OAM info? i.e carried alongside the
> data (sure request for a path trace doesnt fit; but inband
> healthinfo may fit); in such a case OAM info could be carried in something
> like a TLV.
That was the intent. The formatting issue you caught in the bulleted
list made this less than clear.

>     In real deployment, the security of the underlying network may be
>     considered for determining the level of security needed in the
>     encapsulation layer.  However for the purposes of this discussion, we
>     assume that network security is out of scope and that the underlying
>     network does not itself provide adequate or as least uniform security
>     mechanisms for encapsulation.
> I found the above paragraph awkward to read.  How about simplifying:
> "This document assumes that the underlying network does not itself
> provide adequate or at least uniform security mechanisms for encapsulation.
> The authors understand that the underlying network security could provide
> useful input into the security needs of the encapsulation layer but ignore
> it to provide a focus on the discussion."
Let me check whether that was indeed the intent of the text.

>     o  Interaction with packet level security such as IPsec or DTLS
> So would IPSEC not be considered "underlying network security"?
Probably not. If you want to ensure e.g. isolation between different
tenants then you probably want to avoid know plaintext attacks from one
tenant against another. That means that the security associations need
to be different for the different tenants' encapsulated traffic. Hence
the IPsec policy would need to be aware of how the tenants are described
in the encapsulation somehow.

Stated differently, can't just apply IPsec pixie dust. Requires some
real work to specify the details.
> Confidentially is one - but what about integrity of the VNI?
That's captured in the Anti-spoofing bullet.

>        *  The criticality of virtual network isolation depends on whether
>           tenants are trusted or untrusted.  In the most extreme cases,
>           tenants might not only be untrusted but may be considered
>           hostile.
> So would confidentiality then become a requirement to address this?
> It is more readable to make suggestions on each issue on what needs
> to be done.
I think it is too early to tell. For NVO3-like deployment there has some
thoughts, but I don't know if we have the same level of understanding
across encapsulations. And even for NVO3-like deployments it could be
different - a public provider where anybody can get a virtual machine,
vs. isolation between different departments in an enterprise NVO3 network.

>>     o  Our collective IETF experience is that succesful protocols get
>>        deployed outside of the original intended context, hence the
>>        initial assumptions about the threat model might become invalid.
>>        That needs to be considered in the standardization of new
>>        encapsulations.
> So whats the recommendation here? Over-engineer in case something is needed
> later?
At least provide the extensibility mechanisms so it can be added without
too much pain.

> 12.  QoS
>     In the Internet architecture we support QoS using the Differentiated
>     Services Code Points (DSCP) in the formerly named Type-of-Service
>     field in the IPv4 header, and in the Traffic-Class field in the IPv6
> Its been at least a decade since the change, do you really need to say
> "formerly named ToS"?
I dunno. I was told by the LISPers that the RFC 1812 ICMP errors
requirement is not yet implemented (getting more than 8 bytes past the
IP header) and that was 20 years ago.

> Is a ref for NCP needed?
Why not? RFC33 seems to be the most pertinent RFC.

>     Extending a protocol header with new fields can be done in several
>     ways.
>     o  TLVs are a very popular method used in such protocols as IP and
>        TCP.  Depending on the type field size and structure, TLVs can
>        offer a virtually unlimited range of extensions.  A disadvantage
>        of TLVs is that processing them can be verbose, quite complicated,
>        several validations must often be done for each TLV, and there is
> I think if you make such strong comments you need to quantify them.
> A TLV is a formal structure with well defined characteristics. You could
> write efficient code to parse, identify and validate TLVs. How is it
> verbose to process etc?
>>        no deterministic ordering for a list of TLVs.  TCP serves as an
> The reason deterministic ordering would matter is if there's dependencies
> between the TLVs. If that is a huge need, then the document needs to provide a
> sample space or explanation why that is important.
Fair enough; let me check with the co-authors on how to improve those parts.

> The main difference seems to be in the fact that in a list of header
> extensions, the current extension describes the next; whereas in TLVs
> there is no such relationship; otherwise the T in TLV is an extension
> header. One imposes ordering, the other doesnt really.
In the case of IPv6 header extensions there isn't a strong ordering
requirement; there is a recommended order in which to transmit them but
a receiver should be more liberal.
Header extensions seem to be set apart from options/TLV/extensions in
that they use the same next-header space as is used to indicate a
"terminal" header such as TCP.

>>     o  Flag-fields are a non-TLV like method of extending a protocol
>>        header.  The basic idea is that the header contains a set of
>>        flags, where each set flags corresponds to optional field that is
>>        present in the header.  GRE is an example of a protocol that
>>        employs this mechanism.  The fields are present in the header in
>>        the order of the flags, and the length of each field is fixed.
>>        Flag-fields are simpler to process compared to TLVs, having fewer
>>        validations and the order of the optional fields is deterministic.
>>        A disadvantage is that range of possible extensions with flag-
>>        fields is smaller than TLVs.
> Qualify with "much smaller" maybe?
Defer with other TLV comments.

> "a middlebox could first decapsulate, perform some function then encapsulate;
> which means it will generate a new encapsulation header."

Much better - and I've applied your other editorial fixes that I didn't
explicitly respond to.
> Is there a reference to work which says quiet periods (which i am implicitly
> reading that in the text above) can be used to change the hash selection?
> I would think that one needs to closely observe packet trends to make
> such a decision. So please provide some ref to some scholarly or
> engineering work.
I don't know of citations to such work; perhaps my co-authors have some.
I recall reading about using markers, but that was a long time ago.

> I think this section is well done.
> In regards to TLVs, I understand now a little more where the earlier
> comments come from (IMO: you will need to point to a reference to this
> section from the earlier reference).
> Having said that, lets weigh out the pros and cons:
> pro:
> TLVs very flexible - almost give you future proofness in terms of extensibility.
> cons:
> Harder to parallelize in hardware.
> I think the pro side should be driving things.
> I would say to the hardware folks - get busy now!
> I am still unsure why this is hard to do in h/ware given all the benefits.
> At the expense of getting tomatoes thrown at me:
> sounds like there's an extra parsing step of hardware processing to find each
> individual TLVs "fixed offset" and after that you can parallelize.
This is something that should be discussed in RTGWG.


> cheers,
> jamal