Re: [tsvwg] Neal Cardwell's rationale for supporting ECT(1) as an input/L4S signal

[Bob Briscoe <[e-mail]>]

Neal, Jake,

On 12/05/2020 21:55, Neal Cardwell wrote:
> On Fri, May 8, 2020 at 6:02 PM Holland, Jake
> <jholland=[e-mail]> wrote:
>> Hi Neal,
>>
>> Thanks for posting this, it’s a helpful explanation.
>>
>> A couple of points I wanted to respond to:
>>
>> From: Neal Cardwell <ncardwell=[e-mail]>
> ..
>> I wanted to check to make sure I understand:
>> To what extent do you think it's an L4S goal to accommodate the
>> existing installed base of switch hardware that’s currently in use
>> for DCTCP in datacenters?
> I'm not sure if accommodating the existing installed base of switch
> hardware was an explicit goal of L4S or not, but it is a nice property
> that seems to help greatly in making L4S easier to deploy than it
> might otherwise be.
>
>> I thought the existing deployments generally wouldn’t be compliant
>> L4S-compatible dualq devices suitable for general internet traffic
>> anyway, and would continue to need traffic isolation the way they do
>> now.  Is that different from your understanding?
> My understanding is that dualq is not a required component of
> implementing L4S, and definitely would not be required at every hop or
> potential bottleneck along the network path. My understanding is that
> there would be sites that don't want to change the qdiscs on their
> senders/servers, and don't want to change their datacenter switches,
> but would like their connections over the public Internet to be able
> to use L4S.

[BB] Heterogeneous DC networks was actually the original use-case that 
motivated the work that became known as L4S.

Back in 2013, when I worked for BT, most of my work was for BT's finance 
sector customers (BT operated what used to be the Thomson-Reuters 
Radianz network, which is the largest finance network in the world, 
interconnecting all the finance centres and stock markets). We did an 
analysis of the solutions that gave the greatest latency gains with the 
least pain (deployment cost). DCTCP had great potential, but we couldn't 
deploy it because it needed a single flag-day deployment, whiach wasn't 
feasible on this private internetwork operated by hundreds of 
independent administrators. Glenn Judd at Morgan Stanley was using 
traditional Diffserv for isolation [Judd15], but the whole Radianz 
traffic matrix was too unpredictable to configure static partitions.

That's why I started on the problem of coexistence between classic TCP 
and DCTCP [Kuehlewind14], which was just as applicable to the public 
Internet as to private DC networks. These remained the two strongest 
use-cases as we developed the DualQ Coupled AQM within the EU-funded 
RITE project, alongside Koen's team from Al-Lu, who were more focused on 
broadband access, which incidentally led to BT's network architect 
sponsoring the work internally in addition to the finance sector 
business unit.

After I left BT, I lost the link with the finance sector clients, but 
the DC use-case was still very relevant at Simula Research. Indeed, in 
2016 the Curvy RED pseudocode in draft-briscoe-tsvwg-aqm-dualq-coupled 
was picked up for implementation in merchant switch silicon.

However, once I started with CableLabs in 2017, the core L4S team no 
longer included anyone directly working on DCs, which is probably why 
you aren't aware of the DC heritage behind L4S.

____________
Quoting from the Use cases section of the L4S architecture:

    o  Private networks of heterogeneous data centres, where there is no
       single administrator that can arrange for all the simultaneous
       changes to senders, receivers and network needed to deploy DCTCP:

       *  a set of private data centres interconnected over a wide area
          with separate administrations, but within the same company

       *  a set of data centres operated by separate companies
          interconnected by a community of interest network (e.g. for the
          finance sector)

       *  multi-tenant (cloud) data centres where tenants choose their
          operating system stack (Infrastructure as a Service - IaaS)


And quoting from the Scope section of draft-ietf-tsvwg-aqm-dualq-coupled:

    ... it is believed the
    Coupled AQM would be applicable and easy to deploy in all types of
    buffers; buffers in cost-reduced mass-market residential equipment;
    buffers in end-system stacks; buffers in carrier-scale equipment
    including remote access servers, routers, firewalls and Ethernet
    switches; buffers in network interface cards, buffers in virtualized
    network appliances, hypervisors, and so on.



[Kühlewind14] Kühlewind, M., Wagner, D.P., Espinosa, J.M.R. & Briscoe, 
B., "Using Data Center TCP (DCTCP) in the Internet," In: Proc. Third 
IEEE Globecom Workshop on Telecommunications Standards: From Research to 
Standards pp.583-588 (December 2014)

[Judd15] Judd, G., "Attaining the Promise and Avoiding the Pitfalls of 
TCP in the Datacenter," In: 12th USENIX Symposium on Networked Systems 
Design and Implementation (NSDI 15) pp.145-157 USENIX Association (May 2015)

more...

>
>>> - More generally, if there is any problem discovered with the L4S
>>>    experiment, either the algorithm or particular implementations,
>>>    bottlenecks can easily identify L4S traffic and bleach it into Not-ECT,
>>>    and treat it like Reno/CUBIC traffic.
>> To repeat in what's maybe a better thread for it:
>>
>> I agree that bleaching is an option where there’s RFC 3168 trouble,
>> that’s a good point and thanks for mentioning it.
>>
>> I’d consider it a poor choice to land there if there's other options,
>> because it works against the success of existing or under-way
>> deployments of RFC 3168 queues by imposing a new bleaching requirement
>> that may not be trivial for all networks with marking queues
>>
>> It would also be an unfortunate choice to endorse bleaching because it
>> would lose much of the potential value in the code point, as David
>> recently mentioned.
>>
>> But I do agree it would work, and that it’s a worthwhile mitigation
>> that should be mentioned in the L4S docs if we end up not finding a
>> way to support robust 3168 compatibility.
> Yes, I agree with all those points; bleaching is very far from ideal.

[BB] It's important not to change ECT(1) to Not-ECT, and there's no need 
to, as long as you can configure AQMs to treat ECT(1) as Not-ECT.

[I-D.ietf-tsvwg-ecn-l4s-id] says

    "the ECT(1) codepoint MUST NOT be
    changed to any other codepoint than CE"

See also 
https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-10#section-5.4.1.2

5.4.1.2. Exclusion of Traffic From L4S Treatment
...
    The operator MUST NOT alter the end-to-end L4S ECN identifier from
    L4S to Classic, because its decision to exclude certain traffic from
    L4S treatment is local-only.  The end-to-end L4S identifier then
    survives for other operators to use, or indeed, they can apply their
    own policy, independently based on their own choice of locally-used
    identifiers.  This approach also allows any operator to remove its
    locally-applied exclusions in future, e.g.  if it wishes to widen the
    benefit of the L4S treatment to all its customers.




>
>>> - Encapsulation/decapsulation is a widely prevalent and important
>>>    technology today in production networks. With the installed base of
>>>    encap/decap mechanisms, it is likely that for many implementations any
>>>    SCE marking applied to packets would just get stripped off with the outer
>>>    header when decapsulated.
>> I'd be interested in more information about this.  Is there anything
>> publicly known about the deployment footprint of different tunnel
>> implementations in the places likely to deploy L4S dualqs, and
>> whether they're managed by the same entities who control the queues?
>>
>> I'm not sure it changes my position much, but it seems at least
>> useful to know how big a lift it would be to do a tunnel
>> decapsulation change, if that's a necessary piece of a safe
>> approach to L4S that can achieve low latency effectively.
>>
>> (I know Bob gave a long list of tunneling protocols not long ago, but
>> it seems like only some of them are likely to be relevant to the L4S
>> question, and probably only in a reasonably specific set of
>> implementations for at least the early stages...)

All the tunnels and encapsulations that I mentioned are surely highly 
relevant to any "two-output" marking scheme (whether SCE or your 
proposal). I've updated the 2-slide briefing I posted a couple of weeks 
ago to add a third slide about how an AQM applying your proposed marking 
scheme within a tunnel would just get stripped:
http://bobbriscoe.net/presents/2004ietf/2020-04ecn-tunnel-brief.pdf


Actually, my long list was in response to an offlist email from you. So 
I ought to repeat it here for the list:

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
Jake,

[Context: marking ECT(1) -> ECT(0)...]
I'm afraid there are two very strong reasons why we didn't do this:

#1/ That will be stripped by all stds track ECN tunnels and 
encapsulations, e.g. the following that specify ECN processing specifically:

  * RFC3168,
  * IPSec [RFC4301]
  * those tunnels that have since been updated to RFC6040,
  * All IP/UDP/IP encaps [RFC8085]
  * MPLS ECN [RFC 5129],
  * CAPWAP [RFC5415]
  * LISP [RFC6830]
  * VXLAN [RFC7349]

And the following stds track drafts that are widely already implemented 
because they are close to RFC:

  * TRILL (altho that's waiting on the L4S drafts in the RFC Editor
    queue, even tho it's stds track).
  * Geneve [draft-ietf-nvo3-geneve-16]
  * GUE [draft-ietf-intarea-gue-09]

It would also be stripped by all the other stds track encaps that some 
implementers could have written to comply with either of RFC3168 or 
RFC6040, e.g.

  * L2TP [RFC3931]
  * GRE [RFC2784]
  * GTP [3GPP]
  * Teredo [RFC4380]

#2/ [...snip]


Bob
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

> As I mentioned with Roland, here my understanding was based on a
> description of this issue by Bob Briscoe. I'm not sure what the
> original data source is for that issue. (And apologies if I have
> misinterpreted the issue.) But perhaps Bob will have time to fill in
> more of the details or pointers.
>
> Best regards,
> neal

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/