Re: Review of draft-rtg-dt-encap-02 - Encapsulation Considerations

[Erik Nordmark <[e-mail]>]

On 6/9/15 5:25 AM, Carlos Pignataro (cpignata) wrote:
> Hi, RtgWG,
>
> Please find below some review comments on draft-rtg-dt-encap-02.
>
> https://datatracker.ietf.org/doc/draft-rtg-dt-encap/
>
> While this is not a comprehensive fine-tooth-comb review, I hope these (somewhat high-level) comments are useful.
Carlos,
Thanks for the comments and sorry for the delay in responding.

>
> General comments:
>
> Scanning through the document, I have two high-level concerns.
> 	• First, there is really no evident or apparent definition of what “Encapsulation” is. This is not a pedantic comment, but I think it is a root cause of the point that follows.
> 	• Many of the considerations on this document are not applicable to the scoped “encapsulation” per se. Instead, they relate to a specific layer underneath the encapsulation, to a layer that further encapsulates the encapsulation.
>
> For example, the document sets of explaining the need for understanding common requirements and considerations among groups of encaps. In addition to encaps in NOV3, these are also the BIER Header and NSH. Both these can be carried by multiple underlays each with different characteristics. However, the document conflates requirements and characteristics from the encap itself, and intertwines them with characteristics and requirements on the underlay.
You must have read a different document, because draft-rtg-dt-encap 
contains no requirements ;-)
More seriously, trying to come up with a strict definition of 
"encapsulation" or placing general requirements across a broad set of 
approaches would be quite futile since that tends to result in proxy 
discussions which are really about "I don't want that requirement on my 
protocol".

That is why the title and the content of the document "considerations". 
It seems like folks appreciate having captured this broad set of 
considerations in one place, and it might be useful *input* to working 
groups that want to draft their own requirements and protocols.

>
> I think this document would greatly benefit from a cleaner separation of what happens at the encap, versus what is expected, required, or considered from the underlay (transport, network, tunnel, etc). This is particularly exacerbated in the ECMP and MTU sections, but present in others as well.
>
> Take for example ECMP — the BIER header has an Entropy field. Different encaps can provide Entropy fields to be used as a source for entropy, either at the encap (e.g., service) layer or by an underlay. One example is L2TPv3 session id and GRE Key (as per RFC 5640). However, the ECMP sections focus on IP and IP/UDP methods, which frankly are applicable generally and not only to these “Encapsulations”.
Note that the BIER entropy has nothing to do with the "transport" 
entropy; we tried to capture that in the document to avoid confusion.
The BIER entropy is how BIER selects paths. In theory there can be an 
MPLS entropy label in addition for the "transport" entropy.

We've tried to specify how the "transport" entropy fits in, and how that 
relates to using IP/UDP and MPLS transports.
That seems to result useful information in the draft, because when we 
started off the Design Team it wasn't clear to (at least us) how those 
things relate; whether one should have a single entropy field even with 
nested encaps etc.
If there are specific text in the considerations that you find unclear 
or incorrect, then please point them out so we can collectively improve 
the document.

And FWIW that is an ask from Alia and others that the WG add more 
information about MPLS to the document.


What are your specific concerns about the MTU considerations?
AFAICT any device along the path which adds (or grows) a header in a 
packet needs to be concerned about the MTU implications.

>
> And I believe some of these potential disconnects go back to the high-level concern #1, lack of definition/scope of what is an encap (and what is not).
>
> Just to be clear, I am all for inter-layer cooperation and requirement realization. However, if the goal is to understand encapsulation common requirements, then a more clean separation of the encapsulation headers versus a specific transport of said encapsulations will make it easier to understand if those requirements belong in an encap or in an underlay.

FWIW my take is that if the folks working on different encapsulations in 
different WGs consider what it in this document, then we can avoid 
endless discussions of where to draw lines in the sand by working on 
strict definitions.

>
> A smaller one: this document is titled "Encapsulation Considerations” (without any qualifiers), but the abstract narrowscopes it to a specific encaps.
Which encaps? The document focuses on three that are currently under 
development in the IETF. (Perhaps your definitions of "encaps" is what 
the document calls "transport", which would make our discussion rather 
confusing?)
>
>
> Some more specific comments, prefaced with “CMP":
>
> 2.  Overview
> …
>     o  SFC carries service meta-data which might be modified or
>        unmodified as the packets follow the service path.  SFC talks of
>        some loop avoidance mechanism which is likely to result in
>        modifications for for each hop in the service chain even if the
>        meta-data is unmodified.
>
> CMP: This is not complete (incorrect by omission). The SFC Encap is responsible first for Serfice Function Path Identification, and second, for Metadata. See Section 1.3 and 4.1 of https://tools.ietf.org/html/draft-ietf-sfc-architecture-09.
I'll reword it to include the Serfice Function Path Identification part.
>
>     o  SFC is all about carrying service meta-data along a path, and
>        different services might need different types and amount of meta-
>        data.
>
> CMP: It is true that metadata requires extensibility. However the SFC Encap is all about path identification *and* carrying metadata.
> CMP: Also, SFC’s extensibility has to do with OAM and graphs as well.
Ditto.

> CMP: By the way, encaps need to strike a balance between flexibility and performance. This point could be made more explicitly.

Suggestions how and where to do that and not make it sound like 
motherhood and apple pie?
What would you say to make a reader learn something they might not have 
already known about that tradeoff?


>
>     Most of the issues discussed in this document are not new.  The IETF
>     and industry as specified and deployed many different encapsulation
>     or tunneling protocols over time, ranging from simple IP-in-IP and
>     GRE encapsulation, IPsec, pseudo-wires, session-based approached like
>     L2TP, and the use of MPLS control and data planes.  IEEE 802 has also
>     defined layered encapsulation for Provider Backbone Bridges (PBB) and
>     IEEE 802.1Qbp (ECMP).  This document tries to leverage what we
>     collectively have learned from that experience and summarize what
>     would be relevant for new encapsulations like NVO3, SFC, and BIER.
>
> CMP: I think it would be much clearer here to specify terms around what is encap, transport, etc. L2TPv3 for example can run over IPv6 directly, over IP/UDP, etc. Those cases have key differences although the actual L2TPv3 header does not change.

One criteria that I found useful in thinking about that to include and 
exclude from the document was whether there is something non-obvious, or 
something known in one part of the IETF but not known in other parts, 
where sharing that knowledge would result in the IETF producing better 
protocols. If you think strict definitions of encap vs. transport would 
be an aid in such sharing and learning, then we should definitely get 
together (e.g., in Prague) so I can understand.

>
> 3.  Common Issues
> …
>     o  How to provide entropy for Equal Cost MultiPath (ECMP) routing
>
> CMP: And LAGs?
LAG isn't something we tend to standardize in the IETF.
One could add a note saying "Note that the same entropy might also be 
used at layer 2 e.g. for Link Aggregration (LAG)". But that belongs on 
the text and not in the

>
>     o  OAM - what support is needed in an encapsulation format?
>
> CMP: OAM in what context? fault management, performance management, all?

The question is broad, hence "all".
>
> CMP: Also, this section seems to lack an issue of “adaptation” of the payload.

Please explain further.

>
> 4.  Scope
>
>     o  Focus on the class of encapsulations that would run over IP/UDP.
>        That was done to avoid being distracted by the data-plane and
>        control-plane interaction, which is more significant for protocols
>        that are designed to run over "transports" that maintain session
>        or path state.
>
> CMP: I am not sure if the relationship between running over “transports” (which transports?) and control-plane - data-plane complexity is clear — it is not clear to me at least.
This was the scope that the design team set up to improve its 
probability of delivering a useful document on time.
Hence the "scope" section presumably needs to be done as the WG works on 
this document.
>
> 7.  Entropy
>
>     In many cases the encapsulation format needs to enable ECMP in
>     unmodified routers.  Those routers might use different fields in TCP/
>     UDP packets to do ECMP without a risk of reordering a flow.
>
> CMP: Is this an encap-supported requirements, or a requirement of IP/UDP in general, whatever is transported?

I'm not sure I understand your question. When Dino defined the LISP 
header he wanted LISP packets to take advantage of ECMP in unmodified 
routers. That same desire exists for other encapsulations that have been 
and are being defined in the IETF and elsewhere.

Note that the document isn't about requirements!
>
>     the ephemeral port range) plus the outer IP addresses which seems to
>     be sufficient for entropy; using outer IPv6 headers would give the
>     option for more entropy should it be needed in the future.
>
> CMP: And the IPv6 Flow Label and RFC 6438?
The section discusses IPv6 flow label a few paragraphs later.

The design team didn't discuss RFC6438 explicitly, but it seems like the 
approach (from LISP and VXLAN) to use a random UDP source port plus 
outer IPv6 flow label is consistent with RFC 6438. But I'll at least add 
a reference to RFC6438.

>
>     There is some interaction between entropy and OAM and extensibility
>     mechanism.  It is desirable to be able to send OAM packets to follow
>     the same path as network packets.  Hence OAM packets should use the
>
> CMP: This is having the underlying assumption that “OAM” is “OAM packets” as opposed to also fields in data packets. If OAM is in the packet, it fate-shares by definition.

But it is more subtle than that. The difference of OAM frames and the 
frames they should "monitor" must not affect the ECMP calculation. The 
document (tries to) point this out. Please suggest improved text if that 
is unclear.

>
>     Architecturally the entropy and the next header field are really part
>     of enclosing delivery header.  UDP with entropy goes hand-in-hand
>     with the outer IP header.  Thus the UDP entropy is present for the
>
> CMP: This is a very important point, and this clarity in demarcation should be throughout, to understand at which layer different requirements are.
>
> 8.  Next-protocol indication
>
>     Next-protocol indications appear in three different context for
>     encapsulations.
> …
>     Secondly, the encapsulation needs to indicate the type of its
>     payload, which is in scope for the design of the encapsulation.
>
> CMP: I believe that this section should not start assuming an explicit protocol indication. A demux context can provide protocol identification, or it could be explicitly carrier (self-defining package). And perhaps that is the first consideration.
Where does it assume it is explicit? Second paragraph has existing 
examples of both explicit and implicit.
>
>     We
>     have existing protocols which use Ethernet types (such as GRE).  Here
>     each encapsulation header can potentially makes its own choices
>     between:
>     o  Reuse Ethernet types - makes it easy to carry existing L2 and L3
>        protocols including IPv6, IPv6, and Ethernet.
>
> CMP: I do not believe that the options are Ethertype, IP protocol number, or own registry only.
What are the other options? Are you suggesting an implicit 
(session-based) identification?
That would assume some form of session identification.

The document mentions that possibility and ends with "*Encapsulations 
might want to consider the tradeoffs between such more flexible versus 
more fixed approaches."*
> CMP: Nit — repeat IPv6.
Thanks.
>
>     In summary:
>     o  Would it be useful for the IETF come up with a common scheme for
>        encapsulation protocols?  If not each encapsulation can define its
>        own scheme.
>
> CMP: This seems to be a non-sequitur. Why would that be useful?
First of all, it is a question and not a statement.

I think it would make sense for the broader IETF to consider the 
cost/benefit tradeoffs.

There might be some benefits as we define X over Y, Y over X, X over W 
over Y, etc (which we seemed to have done with most existing protocols 
over the last few decades). But there would be some cost to defining 
this namespace across different active WGs.
>
> 9.  MTU and Fragmentation
>
> CMP: This is another key section that could use clarify between encap and transport.
See above.
>
>     In summary:
>     o  In some deployments an encapsulation can assume well-managed MTU
>        hence no need for fragmentation and reassembly related to the
>        encapsulation.
>     o  Even so, it makes sense for ingress to track any ICMP packet too
>        big addressed to ingress to be able to log any MTU
>        misconfigurations.
>
> CMP: This section seems to be conflating MTU with PMTUD. Cleary related, but those are different things. The area of pre-fragmentation vs. post-frag is also underspecified. See e.g., https://tools.ietf.org/html/rfc3931#section-4.1.4

I don't think it is useful to talk about PMTU because the first bullet 
is really about a common interface MTU across the whole underlay 
network. That underlay network will have lots of different paths. Hence 
to make it the text more specific I think one would need to introduce a 
notion of a some "topology MTU" accross the underlay topology. 
Introducing such a brand new concept and term doesn't seem to be 
worth-while in explaining the issue and choices.

This document has a number of considerations; I certainly don't expect 
it to specify all the details of how to handle MTU and frag/reass.

>
>
> 10.  OAM
>
>     In terms of what we have heard from the various working groups there
>     seem to be needs to:
>     o  Be able to send out-of-band OAM messages - that potentially should
>        follow the same path through the network as some flow of data
>        packets.
>
> CMP: It is not clear what are “out-of-band messages” that follow the same path as data packets…

TRILL has an example of this. Might exist elsewhere as well.
>
> CMP: Also, this section seems to imply that “OAM” is “separate OAM packets”, where there is OAM Performance Management (PM) considerations for in-band in-data-packets measurement. Those ought to be considered in the encap layer.
Oops. There is an issue with the nesting of the bullets. Should read:
    In terms of what we have heard from the various working groups there
    seem to be needs to:
    o  Be able to send out-of-band OAM messages - that potentially should
       follow the same path through the network as some flow of data
       packets.
       *  Such OAM messages should not accidentally be decapsulated and
          forwarded to the end stations.
    o  Be able to add OAM information to data packets that are
       encapsulated.  Discussions have been around:
       *  Using a bit in the OAM to synchronize sampling of counters
          between the encapsulator and decapsulator.
       *  Optional timestamps, sequence numbers, etc for more detailed
          measurements between encapsulator and decapsulator.
    o  Usable for both proactive monitoring (akin to BFD) and reactive
       checks (akin to traceroute to pin-point a failure)

Does that make it more clear?

>
>     There can be several ways to prevent OAM packets from accidentally
>     being forwarded to the end station using:
>     o  A bit in the frame (as in TRILL) indicating OAM
>     o  A next-protocol indication with a designated value for "none" or
>        "oam”.
>
> CMP: Or both! An OAM bit with an OAM proto allows for max flexibility.
With all the (hardware and software) implementations having to inspect both?
Anyhow, such details can be worked out in the WGs working on the actual 
encapsulations.

Thanks again for your comments.

I'll submit -00 with the changes I have so far.

    Erik

>
> Hope these help!
>
> — Carlos.
>
>
> _______________________________________________
> rtgwg mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rtgwg