Re: [tsvwg] ECN encapsulation draft - proposed resolution

[Markku Kojo <[e-mail]>]

Hi David, all,

Catching up ...

I'm afraid there is an open/ongoing discussion on this where we have not 
reached concensus. Last message on the topic is here:

  https://mailarchive.ietf.org/arch/msg/tsvwg/AtEI72QCFhOWOn9d6xNcrssTVzs/

Note that while this discussion is much about splitting IP packets to 
smaller fragments/frames/PDUs and reassembling/decoding these smaller 
PDUs back to larger IP packets, it does, however, relate to the problem 
in ECN encapsulation draft on what to do when framing boundaries do not 
necessarily align with packet boundaries. This is because the case when 
boundaries do not align often INVOLVES also the case where IP packets are 
splitted into smaller L2 PDUs (or several smaller IP packets are gathered 
into a larger L2 PDU). So, the solution should be based on the same 
known-to-work method no matter whether we are fragmenting/reassembling IP 
packets or encoding/decoding IP packets to/from L2 PDUs. As the discussion 
referred in the above message is longish, I'll try summarize the 
problem space in the end of this message.

And, then there is also another thread that Bob initiated and I seemingly 
have not replied although promised (I was away from any IETF work in 
April due to family emergency and only resuming now, my apologies). That 
thread is about the (additonal but not independent) problem where L3 
packet and L2 PDU boundaries do not align. The last message on the thread 
is here:

  https://mailarchive.ietf.org/arch/msg/tsvwg/3la3kG5-JLU2OPx3zGxxmhWGYEo/

I will send my notes on that very issue separately tomorrow.

I believe that the best solution to allow ECN encap draft to move forward 
is that the draft does not say anything on the topic except points to a 
new draft (the one that has been envisioned to handle the IP 
fragmentation problem and would include also the handling of not aligned 
packet/frame boundaries) and we initiate such new draft before ECN encap 
draft gets published.

Below I try to summarize the problem with the two suggested paragraphs 
with two SHOULDs.

1. The two paragraphs (SHOULDs) are contradictory: there is no algorithm 
that has been shown to be able to correctly fulfill both requirements 
(please see the first message referred above where I explain why the 
algorithms that Bob has suggested do not work correctly).

2. There is no actual algorithm in the draft for a potential implementor 
that would allow for a proper implementation (nor is there experimental 
evidence showing any such algorithm would work correctly).
With the relatively vague first SHOULD, it is very unlikely anyone would 
get it right, I think.

3. RFC 7141 has been used as the guideline for the new proposed text
    which makes resolving the issue even more problematic because the
    recommendations in RFC 7141 are the origin of the problem on how
    to handle fragmentation/reassembly as well as encapsulation/
    decapsulation of IP packets/L2 frames when there is an AQM
    marking/dropping fragmented/splitted PDUs. As said, this is not
    independent of the problem with not aligned packet/frame boundaries.
    The problems that we have at hand are as follows:

  a) The original paper on RED suggested byte-mode operation where
     byte-mode dropping/marking would adjust drop/mark probability
     of smaller fragments (to have lower probability) such that with
     the same level of bit-congestion the RED AQM would mark/drop
     approximately at the same point a packet for a flow being
     fragmented and for a flow with full-sized packets, i.e., it
     would treat fragmented and non-fragmented traffic fairly.
     That byte-mode drop together with RFC 3168 reassembly logic
     results in fair and correct behavior for Standards track
     congestion control, loss-based CC included.

  b) Recommendation in RFC 7141 to not do byte-mode drop but
     instead use packet-mode drop (with equal drop/mark probability
     regardless of PDU size) will treat fragmented (splitted)
     traffic unfairly, yielding fragmented traffic suboptimal
     performance (as Bob has indicated several times). Therefore,
     packet-mode drop would need an algorithm at reassembly/decapsulation
     that SHOULD approximately preserve the proportion of PDUs(/bytes)
     with congestion indications arriving and leaving.

     However, there is no known algorithm that could do this correctly
     at reassembly/decapsulation as stated in item 1 above.
     More importantly, even if such algorithm existed it cannot
     work with non-ECT AQMs, i.e., with loss-based congestion control
     for which no adjustment of congestion indications can be done
     at the reassembly/decapsulator. But, one can achieve the correct
     outcome in a single place: in the AQM algorithm itself by employing
     byte-mode dropping/marking; it works correctly also for the majority
     of the traffic today, that is, for traffic employing loss-based
     congestion control.

     Moreover, doing adjustment at the reassembly/decapsulator is
     architecturally not very good solution because we would
     need such an algorithm at several places (at receiving
     enpoints, at tunnel egress, at L2 decapsulator) which
     introduces quite unnecessary complexity in several places.

  c) Recommendation in RFC 7141 is not applicable with AQMs that do
     not use probabilistic dropping/marking. E.g., it would result
     in incorrect behavior with CoDel AQM that employs deterministic
     dropping (please see more detailed explanation in the first
     message referred above) and with any potential new AQM that
     does not employ probabilistic dropping/marking.

  d) Although RFC 7141 is BCP, the recommendation in it is not based
     on any deployed mechanism (or at least I am not aware of any such
     best practice) nor on any published/evaluated algorithm that has
     shown to work. On the other hand, there is quite a bit of
     experimental evaluation on RFC 3168 reassembly & byte-mode
     drop/mark (although more high-quality evaluation would be useful).


Thanks,

/Markku

On Tue, 25 May 2021, David Black wrote:

> 
> As draft shepherd and a WG chair, I believe that these drafts resolve the last of the open issues from WG
> Last Call.
> 
>  
> 
> In the next week or two, I will prepare the shepherd writeups and submit these drafts to our AD for further
> review towards IETF Last Call and publication as RFCs.
> 
>  
> 
> Thanks, --David
> 
>  
> 
> From: Bob Briscoe <[e-mail]>
> Sent: Tuesday, May 25, 2021 10:35 AM
> To: Black, David; [e-mail]
> Cc: Donald Eastlake; John Kaippallimalil
> Subject: Re: [tsvwg] ECN encapsulation draft - proposed resolution
> 
>  
> 
> [EXTERNAL EMAIL]
> 
> David, tsvwg list,
> 
> As promised yesterday, we just posted a new rev of ecn-encap-guidelines-16, with text based on your
> (David's) suggestions below. To add the references and to avoid some repetition, I twiddled the order round,
> but otherwise kept the text intact.
> 
> I also took the opportunity to post a new rev of rfc6040update-shim, 'cos I noticed Geneve has been
> published as an RFC. There were also a couple of words edited in my local copy as agreed on the list a few
> months ago.
> 
> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-encap-guidelines [datatracker.ietf.org]
> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-rfc6040update-shim [datatracker.ietf.org]
> 
> Cheers
> 
> 
> 
> Bob
> 
> On 24/05/2021 13:50, Bob Briscoe wrote:
>
>       David, Thx for bringing this one up. See [BB] inline,
>
>       On 22/05/2021 01:02, Black, David wrote:
>
>       On another topic, I believe that I have good news to pass along on the ECN encapsulation
>       drafts.
>
>        
>
>       The current situation is that the 6040update-shim draft is ready for RFC publication to be
>       requested, but there's an open issue in the ecn-encap draft on the contents of this
>       paragraph in Section 4.6 (Reframing and Congestion Markings),
>       https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-encap-guidelines-15#section-4.6
>       [datatracker.ietf.org]:
>
>        
>
>          Congestion indications SHOULD be propagated on the basis that an
>
>          encapsulator or decapsulator SHOULD approximately preserve the
>
>          proportion of PDUs with congestion indications arriving and leaving.
>
>        
>
>       Digging further, this area appears to be dealt with in greater length and detail by RFC
>       7141 (Byte and Packet Congestion Notification) Section 2.4 (Recommendation on Handling
>       Congestion Indications When Splitting or Merging Packets),
>       https://datatracker.ietf.org/doc/html/rfc7141#section-2.4 [datatracker.ietf.org].  The
>       short summary is that the quoted sentence is generally correct with RFC 7141 containing a
>       more comprehensive discussion including an exception.  As RFC 7141 is a BCP, I suggest
>       treating it as authoritative on this matter for now, leaving redesign in this area to a
>       possible future draft (as we did in the 6040update-shim draft wrt RFC 3168 fragment
>       reassembly requirements).
>
>        
>
>       To carry this out, here's an initial ecn-encap draft text change suggestion (begins with
>       last two sentences in second paragraph of Section 4.6):
>
>        
>
>       OLD
>
>             Where framing boundaries do not necessarily align
>
>          with packet boundaries, the following guidance will be needed.  It
>
>          explains how to propagate ECN markings from layer-2 frame headers
>
>          when they are stripped off and IP PDUs with different boundaries are
>
>          reassembled for forwarding.
>
>        
>
>          Congestion indications SHOULD be propagated on the basis that an
>
>          encapsulator or decapsulator SHOULD approximately preserve the
>
>          proportion of PDUs with congestion indications arriving and leaving.
>
>        
>
>          The mechanism for propagating congestion indications SHOULD ensure
>
>          that any incoming congestion indication is propagated immediately,
>
>          not held awaiting the possibility of further congestion indications
>
>          to be sufficient to indicate congestion on an outgoing PDU.
>
>        
>
>       NEW
>
>             Where framing boundaries do not necessarily align
>
>          with packet boundaries, the provisions of Section 2.4 of RFC 7141
>
>          apply to propagation of ECN markings from layer-2 frame headers
>
>          when they are stripped off and IP PDUs with different boundaries are
> 
>    reassembled for forwarding. Those provisions include: "The general
> 
>    rule to follow is that the number of octets in packets with
> 
>    congestion indications SHOULD be equivalent before and after merging
> 
>    or splitting." See RFC 7141 for the complete provisions and related
> 
>    discussion, including an exception to that general rule.
>
>        
>
>          In addition to adhering to the provisions of RFC 7141 Section 2.4,
>
>          the mechanism for congestion indication propagation SHOULD ensure
>
>          that any incoming congestion indication is propagated immediately,
>
>          and not held awaiting possible arrival of further congestion
>
>          indications sufficient to indicate congestion for all of the octets
>
>          of an outgoing IP PDU.
>
>        
>
>       END
> 
> 
> [BB] OK, this is indeed progress.
> 
>
>        
>
>       RFC 7141 (a BCP) would be added as a normative reference.
> 
> 
> [BB] I'll write that up now. And post a revised draft.
> 
> 
> Bob
> 
> 
>
>        
>
>       Comments?
>
>        
>
>       Thanks, --David (as draft shepherd)
>
>        
>
>       David L. Black, Sr. Distinguished Engineer, Technology & Standards
>
>       Infrastructure Solutions Group, Dell Technologies
>
>       mobile [phone] [e-mail]
>
>        
> 
> 
> 
> -- 
> 
> ________________________________________________________________
> 
> Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]
> 
> 
> 
> -- 
> 
> ________________________________________________________________
> 
> Bob Briscoe                               http://bobbriscoe.net/ [bobbriscoe.net]
> 
>