Re: [aqm] Review of draft-ietf-aqm-ecn-benefits-03

[Mirja Kühlewind <[e-mail]>]

Hi Gorry, hi Michael,

btw. you'd think about including a ToC because I think the doc is long enough to 
add one...

otherwise see inline...

>   > For the first part (listing benefits) it might also be good to make
> clear/distinguish who has these benefits. I think all benefits that are
> currently listed are only advantageous for the end host/application. Are
> there any benefits for a network operator? Would it be possible to write
> this document such that I could also use it to point network operators
> to and give them an incentive to enable ECN?
>
> MW/GF: we can only think of one benefit of ECN as currently defined
> (i.e. without basing it on ConEx documents) that obviously targets the
> network operator: making incipient congestion visible (such that it
> could be used e.g. for ConEx). This is addressed in section 3.5. Since
> this is one out of six listed benefits in the table, creating categories
> for end host / application vs. network operator seems unnecessary to us.

Okay. Can you maybe state more clearly in the text (or even organize the 
structure such that is would be clear) who (end/app or network) benefits in each 
section...?

>   > Another high level comment is that you say in the introduction that
> this document "also identifies some potential problems that might occur
> when ECN is used" but then you don't really discuss them. I think to
> show both sides of the coin in this document would make the document
> more useful (and more honest). One point that you mention slightly here
> is that cheating is easier than with loss by not providing the feedback.
> Another point might be fairness between ECN and non-ECN traffic as
> marking will not reduce the queue length and therefore might lead to a
> higher loss rate for the non-ECN traffic instaed. I guess there are
> papers about this; don't have any by hand right now. Are there any other
> problems that should be mentioned?
>
> MW/GF: This was discussed, and we agreed to remove the "drawbacks"
> discussion, to align with the original proposed work. So, we will remove
> this sentence from the introduction (it is in fact a left-over that
> should have been removed before). As for fairness, it seems to us that
> the related thread has concluded without a clear result. In the absence
> of evidence or references we prefer to stay away from hand-waving about
> this matter in the document.

I've looked at the old "Pitfalls when using ECN" section and I agree that this 
section is not really appropriate for this document and therefore was removed 
correctly. But the reason is that this section tries to give recommendations. 
What I would like to see is a section that documents the known problems that 
have been seen in the past. The reason why I would like to see this section is 
that, currently the document tells only positive things about ECN and at the end 
a reader might just wonder why ECN is not deployed if it is so great... I think 
you have to answer this question (by explaining what happen in the past without 
making recommendation for the future).

>
>   > Find more detailed comment by section below:
>   >
>   > Abstract
>   > --------
>   > ...says "...potential benefits when applications enable Explicit
> Congestion Notification (ECN)" -> usually an application cannot able ECN
> because usually it's a system setting...?
>
> MW/GF: Good catch! We'll rephrase this as "..when enabling".

Okay. Or even "... potential benefits for applications (and network operators) 
when enabling ..."...?

>
>
>   > Section 1
>   > ---------
>   > ... says "..separate
>   >   configuration of the drop and mark thresholds is known to be
>   >   supported in some network devices and this is recommended
>   >   [RFC2309.bis]."
>   > RFC2309bis does not recommend different settings, it only say that it
> should be possible have different configuration of both. Further, I
> think this should not only concern THE threshold (whatever this is) but
> usually there are several parameters you might want to set independent
> of each other, e.g. the max mark/drop probability in RED.
>
> MW/GF: Suggested update:
> "While it has often been assumed
> that network devices should CE-mark packets at the same level of
> congestion at which they would otherwise have dropped them, separate
> configuration of the drop and mark conditions. Such separate
> configuration is
> known to be supported in some network devices and this is recommended
> [RFC2309.bis]."

First sentence is missing something but else okay. Second sentence should maybe 
be "Such separate configurability is known to be supported in some network 
devices and this is  recommended [RFC2309.bis]."...?

>
>   > Section 2
>   > ----------
>   > 1) I'm not sure I understand the purpose of this section or maybe
> just the title is wrong. I'm currently seeing this section rather as a
> section that provides the needed background knowledge than is talking
> about deployment. For this purpose I'd put all references and
> potentially a brief summary to other RFC/drafts on ECN in this section
> including RFC2884, RFC4774, RFC5562, RFC6040, RFC6679,
> draft-briscoe-tsvwg-ecn-encap-guidelines and draft-ietf-tcpm-accecn-reqs
> (and rename it).
>
> MW/GF: This section lists requirements for deployment. Suggestion:
> rename to "ECN deployment requirements"

Renaming makes sense then. However I'm also not really sure about requirements. 
It's rather something like 'needed pre-conditions/steps to enable successful 
deployment of ECN'...? Which would then include section 4...? And if this is not 
mentioned to be some kind of background section, would it make more sense to 
move this section after the current section 3 (closer to 4 and 5)?

>
>   >
>   > 2) Second paragraph says:
>   > "Network devices must not drop packets solely because these
> codepoints are used [RFC2309.bis]."
>   > Not sure this is the right document to says this (because currently
> it not seems to be directed to network operator/equipment vendors but
> admins/application developers). However, if it says this, it should also
> say that network devices should not bleach these bits.
>
> MW/GF: suggest: "Network devices must not drop packets solely because
> these codepoints are used or erase these codepoints [RFC2309.bis]."

Yes, but, as stated in a separate mail, the reference probably should be removed 
because [RFC2309.bis] does not give this recommendation.

>
>
>   > 3) First bullet in list says
>   > "A recent survey reported growing support for ECN on common network
> paths [TR15]."
>   > This sounds like TR15 shows that ECN is actually used in the
> Internet. However, TR15 only shows that there are only very few cases
> left where ECN packets are dropped or incorrectly altered. Please
> clarify or remove this sentence here.
>
> MW/GF: suggest: "A recent survey reported that incorrect altering of ECN
> bits or consistent dropping of packets carrying the ECN codepoint is
> rare on common network paths [TR15]."

Okay.

>
>
>   > 4) You could cite draft-bensley-tcpm-dctcp-00 instead of the DCTCP
> Sigcomm paper (or both).
>
> MW/GF: The paper is a stable reference for now.
> But if/when the IETF decides on this, we can add a reference.

Okay.

>
>
>   > 5) I would remove the subsection headings (both 2.1 and 2.2) and just
> add the text there to the main part of the section.
>
> MW/GF: OK
>
>
>   > 6) "An AQM algorithm that supports ECN needs to define
>   >   the threshold and algorithm for ECN-marking."
>   > This is kind of self-redundant and therefore does not really makes
> sense to me to say; of course an algo that supports ECN needs to say
> something about ECN...
>
> MW/GF: We agree, but suggest to keep it nevertheless, it is a hint to
> document authors to not forget that they should specify ECN rather than
> just assuming some default behaviour.

This sentence is already (word-by-word) in [RFC2309.bis] where is makes more 
sense because [RFC2309.bis] talks about all kind of AQM and not only ECN. In 
this document it seems slightly weird because it's anyway only about ECN. I 
would recommend a slight rewording to something like "The used AQM needs to 
support ECN by defining the threshold/parameters and algorithm for ECN marking."

>
>   > 7) You can use TR15 to provide a reference for the first paragraph in
> section 2.2:
>   > "Cases have been noted where a sending endpoint marks a packet with a
>   >   non-zero ECN mark, but the packet is received with a zero ECN value
>   >   by the remote endpoint."
>
> MW/GF: OK, will add the reference there
>
>
>   > 8) I'd move the second paragraph of section 2.2. ("The current..") to
> a potentially new problems section, talking about known/previous
> deployment problems.
>
> MW/GF: the document does not accentuate problems in this way, as a
> result of prior discussion. We therefore think that this paragraph is ok
> in its current place.

Okay. The first sentence of the next paragraph is then redundant with the 
statement of this paragraph. Maybe consider removing this sentence and/or 
merging the two paragraphs.

>
>
>   > 9) I would simply remove paragraph 3-4 of section 2.2 because this
> was basically as already mentioned by referring to 2309bis and rfc6040
> in section 2.1.
>
> MW/GF: we do think these paragraphs add value here: they describe the
> problem in greater detail than the text before, explaining the problem
> here is different - and, we think, better - than just pointing to
> references.

Okay. Please just reread then and check for redundancy with the rest of section 2...

>
>
>   > Section 3.2
>   > ------------
>   > 1) Don't understand why there is a listing here...? Just remove the
> listing and make text out of it...?
>
> MW/GF: This is to help identify the entities that need to collaborate.

The sentence before the listing says: "... using ECN can remove resulting 
delay..." and then there are two bullet points, where one explains how an delay 
is removed but the other doesn't.

The second point says "the transport receiver notes that it has received CE-
       marked packets, and then requests the sender to make an
       appropriate congestion-response to reduce the maximum transmission
       rate for future traffic.".
This is true whenever you use ECN, and is not specific for 'Reducing HOL 
blocking'. So I don't understand why this is listed here...

>
>
>   > 2) The sentence "This also
>   >      avoids the inefficiency of dropping data that has already made it
>   >      across at least part of the network path."
>   > does not belong in this section. This sentence should just be moved
> to section 3.1 (or in an own section) and must be further explained,
> saying that dropping packet at the of the path has already blocked
> resources that other traffic could have used otherwise.
>
> MW/GF: agreed. We will insert it at the beginning of section 3.1.
>
>
>   > Section 3.3
>   > -----------
>   > 1) I'd say this section misses on part of the discussion. It is true
> that if by chance your last  packet(s) get lost ECN can help. However,
> this section reads a little like, with ECN it is save to send packet
> bursts. Which is not true because even if ECN is used by a network
> device, the queue might be too small to hold the whole burst. I believe
> this case happen very often which might be a reason for the higher tail
> loss probability that sometimes is experienced with IW10. Please add
> this point to the discussion.
>
> MW/GF: we agree that we shouldn't say that "with ECN it is ok to send
> packet bursts" - we want to stay away from such general recommendations
> and just state the potential benefit of ECN when it saves the last
> packet of a burst. See our next comment for more:
>
>
>   > 2) I don't really get the point of the second paragraph. First of all
> it is confusion that this paragraph starts which "In addition to
> avoiding HOL blocking,.."; I guess that is left over from a previous
> version of this text...? And then you talk about a connection that is
> currently idle, so why is the performance of this connection that is
> currently not sending anything reduced?
>
> MW/GF: indeed it seems that this paragraph has been mangled during
> updates. To address your item 1 and 2, we suggest the following replacement:
>
> ***
> "While using ECN can never guarantee loss prevention, and thus losses
> at the end of a burst can occur with or without ECN, using ECN can increase
> the chance for that last packet to be ECN-marked instead of dropped.
> This can allow the
> transport to avoid the consequent loss of state about the network path it is
>      using, which would have arisen had there been a retransmission
>      timeout.  Typical impacts of a transport timeout are to reset path
>      estimates such as the RTT, the congestion window, and possibly other
>      transport state that can reduce the performance of the transport
>      until it again adapts to the path."
> ***

Still not fully the point: ECN can't prevent tail loss due to queue overflow, 
however if by chance the last segment(s) was dropped due to a regular AQM-based 
drop decisions, ECN CAN prevent these drops.


>   > 3) I don't understand what "applications that send intermittent
> bursts of data, and rely upon timer-based recovery of packet loss"
> are...? Isn't the transport responsible to not send bursts and care
> about recovery...?
>
> MW/GF: MPEG-DASH traffic for instance, in particular when used over
> non-paced TCP. UDP-based applications too.

I understood which kind of application you were talking about but the sentence 
above seem to be wrong for me, because the application (today) usually choose a 
transport (that might handle pacing and recovery or not) while the application 
might not fully be aware of the consequences of choosing a certain transport e.g 
leading to burst sending. So even though these application largely exist today, 
it might not have been an active choice to send bursts or rely on timers for 
recovery. While this sentence sounds as if the application would require to 
behave this way. Do you know what I mean? Is it maybe possible to simply 
rephrase this sentence? Sorry don't have a good proposal right now...

>
>
>   > 4) For the last paragraph in section 3.3 note that stacks often
> remember RTT measurements for a certain IP address and set the initial
> RTO based on this information.
>
> MW/GF: suggestion: replace:
> ***
> because in this
>      case TCP cannot base the timeout period on prior RTT measurements
>      from the same connection.
> ***
> with:
> ***
> because in this
>      case TCP may not be able to base the timeout period on prior RTT
> measurements.
> ***

Okay.

>
>
>   > Section 3.4
>   > -----------
>   > You still need FEC or some kind of error concealment even if ECN is
> used because you can never be sure that your packet are not get dropped
> (by non-ECN-enable devices or other reasons). Therefore using ECN will
> clearly not reduce complexity. The only thing you can do is to
> potentially reduce the amount of redundancy you send if you know that a
> certain path is ECN enables or don't see losses at the beginning of a
> connection. This can save network resources but actually might not
> improve user experience; in fact the user experience might be worse in
> case there are sudden losses.
>
> MW/GF: suggestion: remove "add complexity and"

Okay.

>
>
>   > Further the text says "negative impact of using loss-hiding
> mechanisms"; I don't really think that FEC has a negative impact as long
> as you've send enough redundancy...? Error concealment might but is used
> less and less. I'd recommend to talk about error concealment only in
> this last paragraph and explain a little further.
>
> MW/GF: error concealment is different from FEC, and it is only mentioned
> in this last paragraph. We suggest to replace "Because this
>      reduces the negative impact of using loss-hiding mechanisms,"   with
> "Because this can reduce the potential negative impact that some
> loss-hiding mechanisms can have,"

Okay. I'd still recommend to use the term 'error concealment' instead of 
'error-hiding' here because you describe error-hiding as FEC or duplication in 
the previous paragraph. Those mechanism only have a negative effect if you send 
to few redundancy while error concealment is used in case you have to few 
redundancy and generates the know artifacts that the user can see.

>
>
>   > Section 3.5
>   > ----------
>   > "Recording the presence of CE-marked packets can therefore provide
>   >   information about the performance of the network path."
>   > Would change to:
>   > "Recording the presence of CE-marked packets in absence of loss can
> therefore provide
>   >   information about the performance of the network path."
>
> MW/GF: ok
>
>
>   > And also say more concretely what is meant with 'performance of the
> network path' -> congestion level or no drops by other middleboxes on
> this path...
>
> MW/GF: This intentionally was kept this vague, but we'd welcome a
> concrete recommendation by a ConEx expert      (indeed "or .. or ..." is
> the problem, there are several possibilities here)

Okay if you intentionally would like to stay vague I'd even use 'state' instead 
of 'performance'. You could say: "... information about the state of the network 
path, such as the current congestion level."...?

>
>
>   > Section 3.6
>   > -----------
>   > 1) I like the section but I would phrase it differently; also it's
> not clear who needs to support what in this case. I'd like to propose
> the following text [not sure about the heading...]:
>   >
>   > "3.6 Opportunity to provide an improved congestion feedback signal
>   >
>   > Loss and ECN marking are both used as an indication for congestion.
> However, while the amount of feedback that is provided by loss should
> naturally be minimized, this is not the case for ECN. With ECN a network
> node could provide richer and more frequent feedback on the congestion
> state of a link which then could be used by the control mechanisms
> implemented in end host to make a more appropriate decision on how to
> react to congestion and to react faster to changes in congestion state.
>
> MW/GF: ok to add this up to here.

Just realized that I didn't state this clearly. These three paragraphs are meant 
to replace 3.6 including 3.6.1 (don't think there is a subheading needed here). 
Therefore the two following paragraph rephrase the third paragraph in section 
3.6.1 (where DCTCP is cited).

Please check again.

>
>
>   > Further while drop-based AQM mechanisms usually operate on a smoothed
> queue length estimation (instead of the instantaneous queue length) and
> therefore slightly delay the feedback signal to avoid unnecessary losses
> in case of transient congestion, this would be not necessary for ECN. If
> congestion is only transient due to short traffic bursts that are active
> for less than one RTT, the congestion signal would reach the sender at a
> time where the congestion is already cleared up. However, instead
> delaying the feedback in the network, the end host could reduce its
> sending rate incrementally based on the extend of congestion (that was
> experienced over e.g. the last RTT) similar as DCTCP. In case if the
> congestion is only transient, the end host would only reduce its rate
> slightly and be able to catch up quickly again. However, in case the
> congestion is persistent, this would help to remove additional delays
> from the network and resolve congestion faster which after all reduces
> the average queuing delay.
>   >
>   > However, current ECN is defined as a 'drop equivalent' in RFC3168. To
> change the semantics of ECN both the AQM in the network nodes and the
> control mechanism in the end hosts would still need to cope with nodes
> or end hosts that rely on the old semantics. Therefore changing the
> semantics can be done more easily in confined environment such as a data
> center. DCTCP is an example that changes both the configuration of the
> used AQM as well as the congestion response in the end host and relies
> on that fact that all nodes in data center are configured the same way.
> [Deployment strategies to change the semantics of ECN in the Internet
> are currently under discussion in the IETF.]"
>
> MW/GF: We think that this goes a bit too far in the direction of hinting
> about implementation and research possibilities that we don't have
> citable proof about (besides: we already refer to DCTCP twice in the
> document, and the 'drop equivalent' semantics are not a MUST in RFC3168).

I'd personally like to see at least one sentence on the point that e.g. RED uses 
a smoothed queue length because it was designed for loss and a congestion 
control that halves its sending rate on congestion. If you design an AQM for ECN 
you could be less conservation. However, its also okay for me to not go into 
more detail here but this is mention already in the current text in the 3. 
paragraph of subsection 3.6.1 that why this is in here.

However, the one point that really should be mentioned is that if you change the 
feedback signal that is provided by ECN, you very likely also have to change the 
congestion control reaction to it and you have to take into account that there 
are non-'updated' senders as well. That was the intention of the last paragraph. 
If you don't want to refer to DCTCP here agian, just take the first sentence (or 
rephrase as you like).

Further, you are right that drop equivalence is not a MUST.
Therefore I would to suggest to rephrase to:
"However, ECN is recommended to be 'drop equivalent' in RFC3168."


>   > 2) I'd move the 1. and 2. paragraph of section 3.6.1 to the
> background/deployment section or to the intro depending what you going
> to do with section 2.
>
> MW/GF: since we intend section 2 to be about deployment requirements
> only, we don't think this fits and would rather leave these paragraphs
> in section 3.6.1.

Sorry, these paragraphs really don't belong here. The subsection heading says 
"Other forms of ECN-Marking/Reactions" while these paragraphs only talk about a 
different way to provide ECN feedback; this does change the marking in the 
router nor the reaction of the sender!

Please consider moving to the intro then (or remove... but I think these things 
should be mentioned somewhere to provide the reference to these docs).

>   > Sections 4 & 5
>   > ---------
>   > First sentence talks about "operational
>   >   difficulties when the network only partially supports the use of ECN,
>   >   or to respond to the challenges due to misbehaving network devices
>   >   and/or endpoints".
>   > I think these are to very different things. Misbehaving network
> devices is a point for a  problems section (where the lesson learned is
> that we didn't think carefully enough about incremental deployment in
> the first place but do now). However, partial deployment is not a
> problem but is a thing we simply have to cope with. The text sound as if
> the goal would be that every router in the whole Internet would at some
> point of time be ECN-enabled. I don't think this will ever happen and is
> also not the goal for me. Routers that are very unlikely to ever get
> congested should no be required to look at the ECN bits or monitor the
> queue length to calculate a mark/drop probability.
>
> MW/GF: we agree, and suggest to replace this sentence with "Early
> deployment of ECN encountered a number of operational
>      difficulties due to misbehaving network devices
>      and/or endpoints."
>
>
>   > However as I said at the beginning I don't really thing that sections
> 4 and 5 belong in this document. If you decided to keep them (you have
> to change the abstract) and I'd recommend to rename them e.g 4.
> 'Incremental Deployment Strategy' or 'Requirements to enable Incremental
> Deployment' and 5. 'Recommendations for enabling ECN in network nodes
> and end hosts'.
>
> MW/GF: we suggest to insert the fact that we discuss deployment in the
> abstract, and rename these sections to 4.: "Incremental Deployment" and
> "Recommendations for enabling ECN"

Okay. Btw. if you add a ToC, I don't think you really need the summary table at 
the end of section 5 anymore.

And please also change the following sentence in section 4:
"This section describes transport mechanisms that allow ECN-enabled
    endpoints to continue to work effectively over a path with partial
    ECN support."
(by replacing 'partial ECN support' with something like 'misbehaving network 
nodes'...?)

Thanks!
Mirja


>
>
>   > I hope that's helpful! Let me know if you have any questions!
>   >
>   > Mirja
>   >
>   >
>
>
> Thank you very much,
>
> Michael & Gorry
>

-- 
------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Communication Systems Group
Institute TIK, ETH Zürich
Gloriastrasse 35, 8092 Zürich, Switzerland

Room ETZ G93
phone: [phone]
email: [e-mail]
------------------------------------------