Re: [Rtg-dt-encap-considerations] Some comments on draft

[Tom Herbert <[e-mail]>]

On Mon, Mar 9, 2015 at 10:40 AM, Erik Nordmark <[e-mail]> wrote:
>
> Thanks for the careful review. I've applied the edits except as noted below.
>
> On 3/7/15 3:59 PM, Tom Herbert wrote:
>>
>> Abstract:
>>
>> in the case when->in the case that
>>
>> Section 2
>>
>> The encapsulation protocols typically->Encapsulation protocols typically
>>
>> And NIC hardware->NIC hardware
>>
>> Section 4
>>
>> 2nd bullet is a lttle wordy. I think this is saying we assumed mostly
>> stateless encapsulation protocol
>
>
> "stateless" means different things to different readers.
>
> But it can be made shorter. How about:
> <t>Focus on the class of encapsulations that would run over IP/UDP. That was
> done to avoid being distracted by the data-plane and control-plane
> interaction, which more significant for protocols that are designed to run
> over "transports" that maintain session or path state.</t>
>
Better. "which is more significant"

>>
>> Section 5
>>
>> The goal to->The goal is to
>>
>> the need to be aware->they need to be aware
>>
>> The section on IPv6 flow label is still a little conservative I think.
>> We already have RFC 6438 that described use of flow labels with
>> tunnels. Also, this might be better positioned as a way to provide
>> additional entropy (14 bits for tunnels is going to be a little light
>> in some contexts).
>
>
> I reworded it after the call on Thursday. The last sentences say:
> Currently the leaning is towards recommending using the UDP encaps for both
> IPv4 and IPv6 underlay. The IPv6 flow label can be used for additional
> entropy if need be.
>
Okay. Generally, I do think we're going to want to use the flow label
for something within DCs, 20 bits is just too much opportunity to
waste.

>
>>
>> Section 8
>>
>> some way to indicate which encapsulation header will follow... not
>> clear. This saying: some way to indicate next protocol?
>
> It is really about the encaps, but the preceeding header could also carry
> non-encaps.
> How about wording it as:
> The transport delivery mechanism for the encapsulations we discuss in this
> document need some way to indicate which encapsulation header (or other
> payload) comes next in the packet.
>
Maybe just generically call it the type of the payload?

>>
>> Bullet 2. I don't understand the part about "bringing in protocol
>> numbers that don't make sense".
>
>
> It is trying to capture the questions like would it make sense to run OSPF,
> ICMPv6, etc directly on top of the encaps.
>
> Is it more clear to say:
> "that would never be used directly on top of the encaps protocol"?
>
I think there is a difference between something that doesn't make
sense and something that doesn't work or is broken. I don't see
anything fundamentally broken about encapsulation of ICMPv6 over UDP,
this is already supported in foo-over-UDP for Linux. Maybe say "would
not normally be encapsulated"...

>
>>
>> Section 10
>>
>> A single marker in the encaps... there was discussion in nvo3 such a
>> plan and it had some problems By using a single bit the epochs are
>> forced to be rather coarse. Other problems with part about using a s
>> single bit to do 1-way latency. I think this paragraph should be
>> removed or expanded with more detail.
>
> I'm puzzled because earlier versions suggested encaps reserving room for
> multiple bits and you convinced me that didn't make any sense. Now you seem
> to be saying that one bit isn't sufficient and there has to be multiple
> bits.
>
I believe my suggestion was to allocate a single bit in GUE that
provides a field that could hold generation numbers for more
granularity in an epoch. The OAM proposal on the nvo3 list defined the
algorithm with only the two bits of input to handle both the lost
counters and 1-way delay. The constraints of only using bits puts
constraints on the algorithm and the quality/granularity of
measurement.

> I'll preface it and reword as "There has been suggestions that one (or more)
> marker...".
>
Okay.

>>
>> Section 11: Still needs to be cleaned up. I will try to do that.
>>
>> Section 13: Data centers may be very highly optimized for their own
>> internal traffic by assuming that all internal traffic in under a
>> specific congestion control. That hash allowed us to size and carve
>> buffers in very specific ways. With cloud, we introduce 3rd party
>> stacks, that may implement CC, but not one which is conformant with
>> internal traffic. This is creating problems and may be the motivation
>> for true cc at encap layer
>
> What edit are you suggesting?
>
Just pointing out that circuit breaker may no be sufficient in some instances.

>>
>> Last paragraph: Rate limiting is not the same thing as congestion
>> control (unless there's very active management of rate)
>
> What edit are you suggesting?

Same as above, rate limiting as a means of congestion control may not
be sufficient in some cases.

>>
>>
>> Section 14
>>
>> There is little value in the portion of the UDP checksum... true in
>> terms of being redundant to an encapsulated checksum, however there is
>> value in leveraging the outer checksum to validate inner one using
>> common NIC offload features.
>
> What edit are you suggesting?

The above wording.

> This will be mentioned in the hardware friendly offload text that you
> provided.
>
Okay.

>>
>> For description of partial checksum maybe add: This checksum would
>> cover the encapsulation header and a pseudo header that includes at
>> least the IP addresses and ports.
>
> Doesn't "the same partial-checksum concept as UDP-Lite" already say that?
>
I am assuming the the header checksum does not automatically include
the whole UDP header, so the UDP ports need to be part of the pseudo
header checksum (e.g. draft-herbert-guecsum-00)
>>
>> For NAT problem this can be solved with an additional field that holds
>> the checksum of the original IP addresses.
>
>
> I don't think we need to propose solutions here. Pointing out the issue
> should be sufficient.

Okay

>>
>>
>> Section 17
>>
>> Once again the idea of spraying packets of single flows has been brought
>> up:
>> https://www.cs.purdue.edu/homes/kompella/publications/infocom13.pdf
>>
>> It's probably more correct to say that TCP is optimized for in order
>> delivery, but can handle ooo. Some other protocols (non-IP) might have
>> much more difficulty.
>
> I don't see where that text would fit into the current text.

No, but this may come up at some point as the counter point to all the
efforts to strictly maintain compatibility with ECMP.

>>
>>
>> Section 18
>>
>> Keep encap headers small-- can we be more specific on what small means?
>
> 42 bytes?  ;-)
>
I was hoping for 128 bytes :-)

> But perhaps we don't understand the question yet.
>
>    Erik
>
>>
>> _______________________________________________
>> Rtg-dt-encap-considerations mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/rtg-dt-encap-considerations
>>
>