Re: [tsvwg] I-D Action: draft-ietf-tsvwg-ecn-l4s-id-06.txt

["Scheffenegger, Richard" <[e-mail]>]

Forwarding to the entire list:

Hi Bob,

I’m ok with all the answers you provided.
I would like to encourage that draft to the attention of implementers of
current RDMA hardware, for them to think about L4S implications 😊

Regarding appendix B.1:

My suggestion would be to mention DSACK as another technique (currently
only used by Linux) to mitigate the effect of spurious CC reactions,
when it turns out that the reaction was unnecessary.

With RFC6675 ("New" SACK), there is a slightly higher chance of
triggering loss recovery then three consecutive, reordered packets. For
regular non-SACK flows and the old three-dupack rule, you are correct
that singular reorderings are not an issue, ever.




 >> Apdx A.1.7, ~para 5
 >>
 >> As another example, LACP restricts individual flows to use only
 >> one specific constituent link - effectively restricting the
 >> maximum bandwidth attainable for a single flow to no more than
 >> the bandwidth of a single link - even when the majority of the
 >> remaining links is idle. (this is a common misconception by
 >> enterprise network operators, who believe LACP can "scale up"
 >> the bandwidth available for  a single mount)

 > Yup. As long as LACP is still being used e2e, this single link
 > constraint would have to remain. The L4S requirement about counting
 > in time units might be applied to whatever e2e protocol replaced LACP
 > in future, then it would scale with the transfer rate.

LACP is p2p not e2e 😊. And many devices already allow a round-robin
load balancing (some devices really don’t like this though – older
silicon switching engines usually). TCP RACK  /time based loss detection
would allow the use of RR load balancing for LACP (ethernet), to
actually increase the single-flow throughput across LACP trunks . Sorry
for not making this point more clear.


[removed invalid objection discussed privately]

From: Bob Briscoe <[e-mail]>
Sent: Montag, 25. März 2019 01:49
To: Scheffenegger, Richard <[e-mail]>;
[e-mail]; [e-mail]
Cc: [e-mail]; 'Wesley Eddy' <[e-mail]>
Subject: Re: Review on tsvwg of one of the L4S WG drafts

NetApp Security WARNING: This is an external email. Do not click links
or open attachments unless you recognize the sender and know the content
is safe.

Richard,

Thx for your obviously v thorough review. Inline...
On 24/03/2019 12:52, Scheffenegger, Richard wrote:
Review of draft-ietf-tsvwg-ecn-l4s-id:

Overall, a well-written document. However, I found the following nits,
clarification questions that may need to be answered by Mellanox, before
referenced in the draft, and one incorrect claim about the CE
classification into the L4S Queue:



Nit: I understand Bobs urge to use the British English (borrowed from
French?) centre instead of center. However, at least in "Data Center
TCP" I think the spelling should be US-english.
OK


Section 1.2:

L4S:

May need to solicit input from Mellanox around their RoCEv2 CC mechanism
DCQCN - if this would also fall - and be interoperable with - the L4S
definition. Based on the publicly available info, they designed DCQCN to
work along with instantaneous queue marking, and a variable-extent
feedback mechanism (inspired by QCN). However, I've not seen and
independent studies where DCTCP and DCQCN would run in the same Q.

(RDMA is currently en-vogue in DC environments; Mellanox doing the dev
on RoCE behind closed doors doesn't really inspire too much confidence
in the outcome with me though.
I don't think you're asking me to do anything here. Whatever the outcome
of this question, it doesn't alter what section 1.2 defines L4S to be,
or what it should be.

RoCEv2 is a proprietary protocol. And it's only used in controlled
environments. If it falls under the definition of L4S given here, fine.
If it doesn't, also fine. Whether or not it is compatible with this
definition of L4S does not stop it being used in the same controlled
environments as it is today. Or am I missing something?




Section 5.2.

Cross ref link not yet exists.
Thx for catching this.


Section 5.3

Is text around this eventuality really required in the draft? Wouldn't
long-term identification of flows beyond a single packet be outside the
scope  normally? Just wondering if this complicates the draft more than
it helps.
An FQ implementation with L4S AQM in some queues and Classic in others
is a valid alternative to the DualQ Coupled AQM.

In the stateless case it's straightforward: ECT(1) identifies a packet
as L4S.
But with flow state, we need to define what an L4S flow is, which
implies we have to deal with the case where there's a mix of ECT(0) and
ECT(1) in the flow.




Section 5.4

Another example of a protocol that would benefit from low-latency
treatment, but is low volume would be NTP, and perhaps LDAP (but the
latter can be voluminous at times).
I've added them both.



Regarding ECN++ - references to it are currently only in the appendices.
It appears to me, that and successful operational deployment of a L4S
transport might be dependent on this, to warrant mentioning it somewhere
in the body of the draft too?
The reason it's only in the Appendix is because the Prague requirements
are split into mandatory requirements and optional performance
improvements. Only the mandatory reqs are in the body.

Altho a performance improvement is important, it doesn't need to be
mandated (cos there's no interop implication or downside so it's a
no-brainer without having to be a requirement).




Apdx A.1.7, ~para 5

As another example, LACP restricts individual flows to use only one
specific constituent link - effectively restricting the maximum
bandwidth attainable for a single flow to no more than the bandwidth of
a single link - even when the majority of the remaining links is idle.
(this is a common misconception by enterprise network operators, who
believe LACP can "scale up" the bandwidth available for  a single mount)
Yup. As long as LACP is still being used e2e, this single link
constraint would have to remain. The L4S requirement about counting in
time units might be applied to whatever e2e protocol replaced LACP in
future, then it would scale with the transfer rate.


Para6: type "ussed".
Thx
Note that the wording might incorrectly imply that only RoCE can be used
for RDMA, while at least one IETF alternative exists (iWARP). Also, I
believe the extremely strict in-order requirement was with RoCEv1, while
RoCEv2 did learn from real-world deployments where in-order and
loss-less delivery is not always attainable... Again, something to get
Mellanox to comment about.
Good point. So I could say:
    Also note that in some controlled environments no reordering is
    tolerated by some transports (e.g.  RoCEv2 used for RDMA, in
contrast to iWARP),




Apdx B.1

 >> A reference to DSACK could be made here - while no stack (perhaps
 >> Linux does) performs cwnd unwinding in case of a DSACK for fast
 >> retransmissions (more stacks to unrolling of RTO state changes
 >> when DSACK is found), this would be a mechanism which also mitigates
 >> the problem of spurious retransmissions.

 > What point are you wanting me to make here? I'm not sure a point
about > DSACK would be relevant to the thread of the argument here.
 >
 > Regards and thanks again...
 >
 >Bob





Best regards,
    Richard


Am 12.03.2019 um 01:01 schrieb [e-mail]:
>
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the Transport Area Working Group WG of the IETF.
>
>          Title           : Identifying Modified Explicit Congestion Notification (ECN) Semantics for Ultra-Low Queuing Delay (L4S)
>          Authors         : Koen De Schepper
>                            Bob Briscoe
> 	Filename        : draft-ietf-tsvwg-ecn-l4s-id-06.txt
> 	Pages           : 40
> 	Date            : 2019-03-11
>
> Abstract:
>     This specification defines the identifier to be used on IP packets
>     for a new network service called low latency, low loss and scalable
>     throughput (L4S).  It is similar to the original (or 'Classic')
>     Explicit Congestion Notification (ECN).  'Classic' ECN marking was
>     required to be equivalent to a drop, both when applied in the network
>     and when responded to by a transport.  Unlike 'Classic' ECN marking,
>     for packets carrying the L4S identifier, the network applies marking
>     more immediately and more aggressively than drop, and the transport
>     response to each mark is reduced and smoothed relative to that for
>     drop.  The two changes counterbalance each other so that the
>     throughput of an L4S flow will be roughly the same as a 'Classic'
>     flow under the same conditions.  However, the much more frequent
>     control signals and the finer responses to them result in ultra-low
>     queuing delay without compromising link utilization, and low delay is
>     maintained during high load.  Examples of new active queue management
>     (AQM) marking algorithms and examples of new transports (whether TCP-
>     like or real-time) are specified separately.  The new L4S identifier
>     is the key piece that enables them to interwork and distinguishes
>     them from 'Classic' traffic.  It gives an incremental migration path
>     so that existing 'Classic' TCP traffic will be no worse off, but it
>     can be prevented from degrading the ultra-low delay and loss of the
>     new scalable transports.
>
>
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/
>
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-06
> https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-ecn-l4s-id-06
>
> A diff from the previous version is available at:
> https://www.ietf.org/rfcdiff?url2=draft-ietf-tsvwg-ecn-l4s-id-06
>
>
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
>
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>