Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt

[Bob Briscoe <[e-mail]>]

Yaakov,

This time adding pointer to all my nits and 
including the list, which I hadn't noticed wasn't 
cc'd when I replied to all. More inline within my own previous email..

At 14:02 14/08/2014, Bob Briscoe wrote:
>To: Yaakov Stein <[e-mail]>
>From: Bob Briscoe <[e-mail]>
>Subject: RE: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
>Cc: "Andrew G. Malis" <[e-mail]>, BOCCI 
>Matthew <[e-mail]>, 
>"Black, David" <[e-mail]>
>
>Yaakov,
>
>I have just re-read it thoroughly. I have 
>noticed about two dozen editorial nits where 
>correcting them would not change the intended 
>sense (altho sometimes the sense needs to change 
>to say what we intended!). Examples:
>- Unexpanded abbreviations (David's suggestion would help, but there are more)
>- Missing words, tautologies, random punctuation problems etc
>- Uses of the wrong word, even tho we know what you mean
>- The caveat about dynamics vs steady state 
>needs to shift down a para, so that it follows 
>into the para that says what to do about dynamics.
>
>Should I take the token, and edit the XML 
>directly, then you can see a diff to check if 
>you want to accept my suggestions?

I've provided xml source of my suggested edits (and a diff) here:
<http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-DIFF03a-02.html>
<http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-03a.txt>
<http://www.bobbriscoe.net/projects/2020comms/pwe3-congcons/draft-ietf-pwe3-congcons-03a.xml>

I'm not implying that this is now the master 
version - it's just an easy way to describe edits 
(and you can use it as the basis for a new master 
if you want). Similarly to David, pls take my 
comments as co-author into consideration in parallel with the WGLC responses.

I couldn't alter the figures. I would ask for the following changes:
* All Figs: s/PLR/PLP/
* Figs 1-4:
   - s/delay/one-way delay.
* Fig 1: max at 10 not 50 on y axis
* Figs 5-8:
   - s/safe/undeniably safe/
* Figs 7-8: max at 0.5% on x axis
* Fig 9: s/Pd/PLP/

I incorporated those nits I could see on the list 
(Sasha, Stewart & David), some of which I had noticed myself.

>Below are six thoughts that go slightly beyond 
>'editorial'. 1-4 could be considered 
>'editorial'. But 5&6 are the more significant "techno-political" points.

Suggested resolutions of these six bigger issues 
are now also in my xml suggestion above too.


>1. The phrase "mechanisms for provisioning PWs 
>in service provider networks" is too opaque (the 
>PWE audience might understand this scoping 
>sentence, but a L4 reader might not understand 
>what we're excluding here, which could make them suspicious).
>CURRENT:
>    Mechanisms for provisioning PWs
>    in service provider networks are well understood and will not be
>    discussed further here.
>SUGGESTED:
>    The case where the service provider network 
> arranges sufficient capacity for a PW is well 
> understood and will not be discussed further 
> here. The concern is over PWs deployed 
> independently of the service provider network's 
> traffic engineering or capacity planning.
>
>2. The justification for using OWD of a few milliseconds misses the point.
>"  The one-way delay
>    of a native TDM service consists of the physical time-of-flight plus
>    125 microseconds for each TDM switch traversed; and is thus very
>    small as compared to typical PSN network-crossing latencies.
>"
>The 125us per switch is irrelevant. The point 
>that needs making is that PWs are generally 
>deployed with a short physical time of flight.
>
>3. Related point: the captions on the graphs do 
>not clearly indicate that they are OWD, not RTT. 
>This could be clarified in the text, or on the graphs.
>
>4. Change packet loss rate (and PLR) to packet 
>loss probability throughout, including on the 
>graph axes. PLR is an ambiguous term, even 
>though it is widely used in the industry. Packet 
>loss rate could be measured in packet per 
>second, whereas packet loss probability is 
>unambiguously dimensionless. Indeed, in the 
>appendices, packet loss rates with respect to time are calculated.
>
>5. Rather than David's "reasonable" I suggest we 
>use the phrase "unambiguously safe" not just 
>"safe" for the headings of Figs 5-8 and in the 
>text, as below. Otherwise, we are implying that 
>it is unsafe outside the region.

I thought of a better phrase: "undeniably safe" 
instead of "unambiguously safe".
I slightly prefer this over these other possibilities:
- unarguably safe
- indisputably safe
- unconditionally safe

>CURRENT:
>    One may view this condition as defining an operating envelope for a
>    TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
>    causes no more congestion than that TCP flow would.  Under this
>    condition it is safe to place the TDM PW along with congestion-
>    responsive traffic such as TCP, without causing additional
>    congestion.  on the other hand, were the TDM PW to consume
>    significantly more bandwidth a TCP flow, it could contribute
>    disproportionately to congestion, and its mixture with congestion-
>    responsive traffic might be inappropriate.
>SUGGESTED:
>    One may view this condition as defining an 
> unambiguously safe operating envelope for a
>    TDM PW, as a TDM PW that occupies no more bandwidth than a TCP flow
>    causes no more congestion than that TCP flow 
> would.  This does not draw us into a debate 
> over whether TCP-friendliness is a valid 
> constraint; it simply says that there can be no 
> question that a PW is safe if it does no more 
> harm than a single TCP flow. Under this
>    condition it is unambiguously safe to place 
> the TDM PW along with congestion-
>    responsive traffic such as TCP.  On the 
> other hand, were the TDM PW to consume
>    significantly more bandwidth a TCP flow, it could contribute
>    disproportionately to congestion, and its mixture with congestion-
>    responsive traffic might start to cause concern.
>
>6. The analysis in Appendices A & B needs to 
>mention that loss bursts arise primarily from 
>tail drop buffers, and the introduction of 
>active queue management (AQM) makes losses more 
>independent of each other (less bursty). (I have 
>references showing evidence of this.) In the 
>comparisons with TCP, we have assumed 
>independent losses anyway, because they are the 
>worst case (and fortunately they are easier to 
>analyse). On a similar point, in appendix B, 
>where we say "(bursty packet loss was also 
>simulated but is not reported here)", we ought 
>to give a reason otherwise readers will be 
>suspicious. A good reason would be that uniformly random is the worst case.
>
>I've also responded to a couple of David's 
>suggestions inline (I agree with the others)...

I did /not/ incorporate the more extensive text 
requested/suggested by Stewart/David. I didn't 
want to step all over your call as editor on this.

As you'll see my responses to David's points are 
still inline below. I agree with them all except:
* I don't think we should go into any detail on a 
circuit-breaker mechanism in this draft, which 
should stay on the purely informational high ground.
* Also I don't think we need to recommend a PW 
should prefer larger packets; given we've just 
shown there isn't a problem, we don't need to propose any rememdy.
* I'm not convinced refs for Security 
Considerations are necessary, but I wouldn't mind if you added them either.


Cheers



Bob




>At 17:44 13/08/2014, Yaakov Stein wrote:
>>Bob told me on Friday of the IETF that he was 
>>going to look over the final version and comment.
>>
>>Bob – ping ?
>>
>>Y(J)S
>>
>>From: Andrew G. Malis [mailto:[e-mail]]
>>Sent: 13 August, 2014 17:02
>>To: Yaakov Stein
>>Cc: Bob Briscoe; [e-mail]; BOCCI Matthew; Black, David
>>Subject: Re: [PWE3] WGLC comments on: draft-ietf-pwe3-congcons-02.txt
>>
>>Yaakov,
>>
>>I still haven't heard from Bob, but you should 
>>incorporate David's updates into a new 
>>revision, and I'll use that version for my final Shepherd's review.
>>
>>Thanks,
>>Andy
>>
>>On Mon, Aug 11, 2014 at 11:16 AM, Andrew G. 
>>Malis <<mailto:[e-mail]>[e-mail]> wrote:
>>Actually, I'm going to end the LC as soon as I 
>>hear back from Bob on the IPR call. I sent him a reminder earlier this morning.
>>
>>Cheers,
>>Andy
>>
>>On Sun, Aug 10, 2014 at 9:53 AM, Andrew G. 
>>Malis <<mailto:[e-mail]>[e-mail]> wrote:
>>David,
>>
>>That's perfect, thanks! I'm going to end the LC tomorrow.
>>
>>Cheers,
>>Andy
>>
>>On Sat, Aug 9, 2014 at 10:32 PM, Black, David 
>><<mailto:[e-mail]>[e-mail]> wrote:
>>Following up on this ...
>>
>> > As we're now moving quickly to try to complete this draft , I agreed
>> > w/Yaakov that my author comments would be 
>> sent in as WG Last Call comments ...
>> > ... so here they are:
>>... with some suggested new text ...
>>
>> > Major issues:
>> >
>> > The reference to a transport circuit breaker 
>> in section 3 is a bit optimistic
>> > and high level:
>> >
>> >       We can accomplish this by
>> >       employing a transport circuit breaker, by which we mean an automatic
>> >       mechanism for terminating a flow to prevent negative impact on other
>> >       flows and on the stability of the network
>> >       [I-D.fairhurst-tsvwg-circuit-breaker].
>> >
>> > This should be accompanied by at least an example of how to do this - past
>> > discussion has suggested carrier monitoring of delivered service quality
>> > in order to shut down PWs that are experiencing loss rates that result in
>> > unacceptable service quality (as defined for us by the ITU ...).  I'd also
>> > like to add some text to suggest that operators use this sort of circuit
>> > breaker practice when TDM PWs compete with elastic traffic.
>>After this text in Section 3:
>>
>>    We can accomplish this by
>>    employing a transport circuit breaker, by which we mean an automatic
>>    mechanism for terminating a flow to prevent negative impact on other
>>    flows and on the stability of the network
>>    [I-D.fairhurst-tsvwg-circuit-breaker].  Note that a transport circuit
>>    breaker is intended as a protection mechanism of last resort, just as
>>    an electrical circuit breaker is only triggered when absolutely
>>    necessary.
>>
>>I suggest adding:
>>
>>    For TDM pseudowires, such a circuit breaker can be realized via
>>    operator monitoring of drop rates and consequent on the quality of
>>    TDM service delivered and shutting down TDM pseudowires that persistently
>>    fail to deliver acceptable TDM service (as defined by [G775] and
>>    [G826]).  This document explains why a TDM PW that is causing congestion
>>    problems via interference with elastic traffic cannot be delivering
>>    acceptable TDM service (see the rest of this Section, plus Appendices
>>    A and B).
>
>I don't think we should go there. We need to 
>keep this draft firmly as informational. If 
>someone thinks a mechanism is needed because of 
>the information in this draft, then they can write about a mechanism.
>
>
>>And breaking the paragraph after the added text.
>>
>> > Figure 1 and Figure 2 indicate that based on using the TCP throughput
>> > equation as a proxy for potential impact on elastic traffic (which is what
>> > we're doing in this draft), for the same 
>> fixed bandwidth PW, a larger segment
>> > size is less likely to cause impacts.  I'd like to add a suggestion to use
>> > the larger PW segment sizes as long as they're consistent with delivery of
>> > the service.  There's been a related 
>> (long-to-the-point-of-ratholed) debate
>> > over in the Transport Area about whether packet-rate-limited/congestible
>> > forwarding is still relevant.  My view is that such equipment exists, and
>> > hence that this is a relevant consideration for this draft, and would not
>> > mention that issue here.
>>Ignoring the rathole invitation ;-).  This is still in section 3, after:
>>
>>    We see that in general for large packet sizes, short delays, and low
>>    packet loss rates, the TDM pseudowires consume much less bandwidth
>>    than TCP would under identical conditions.  Only for small packets,
>>    long delays, and high packet loss ratios, do TDM PWs potentially
>>    consume more bandwidth, and even then only marginally.
>>
>>I suggest adding:
>>
>>    Comparing figures 1 and 2, use of a larger segment size for a T1 or E1
>>    TDM PW moves the operational behavior of the PW away from the boundary
>>    at which the TCP throughput equation 
>> suggests that undue interference with
>>    elastic traffic may start to occur.  We suggest that larger segment
>>    sizes be preferred for T1 and E1 TDM PWs, but note that other
>>    considerations may favor smaller segment sizes (e.g., jitter buffer
>>    size, bounding the impact of an isolated packet drop event).
>
>There is no need to recommend any remedial 
>action, when we have just shown there is not a problem.
>
>Further, the "marginal problem" only applies 
>when the TCP traffic uses packets smaller than 
>128B, which we only used as a hypothetical 
>worst-case for the analysis. But in practice 
>this doesn't happen with long-running TCP flows 
>- they use packets that are as large as possible 
>(irrespective of what the PW uses). So there really is /no/ problem to remedy.
>
>
>>and again, breaking the paragraph after that added text.
>>
>>I hope that latter sentence ("We suggest that ...") is acceptable to the WG.
>>
>> > Minor issues:
>> >
>> > Section 2 is about Ethernet PWs that carry elastic traffic.  Should others
>> > be mentioned as also covered by this analysis - e.g., ATM and Frame Relay
>> > PWs carrying elastic traffic?
>>After:
>>
>>    In this section we consider Ethernet PWs that primarily carry
>>    congestion-responsive traffic.  We show that we automatically obtain
>>    the desired congestion avoidance behavior, and that additional
>>    mechanisms are not needed.
>>
>>I suggest adding:
>>
>>    This discussion is applicable to any PW that carries congestion-
>>    responsive traffic (e.g., ATM and Frame Relay PWs); Ethernet PWs are
>>    used here for clarity, as they are a common example.
>>
>> > Section 2 does not cover Ethernet PWs 
>> carrying inelastic traffic - that would
>> > be unusual, but should it be explicitly noted as out of scope, or just not
>> > discussed (as is the case for the current text)?
>>I suggest doing nothing here, and planning to deal with this concern
>>if it should arise later (e.g., IETF Last Call or IESG Evaluation).
>>
>> > Section 3 should say that the reason the draft focuses on TDM PWs is that
>> > they're the inelastic PWs that are likely to 
>> be carried over IP in practice.
>> > My understanding is that other potentially inelastic PWs are MPLS-only
>> > in practice.
>>After:
>>
>>    Inelastic PWs, such as TDM PWs ([RFC4553][RFC5086][RFC5087]), are
>>    potentially more problematic than the elastic PWs of the previous
>>    section.  Being constant bit-rate (CBR), TDM PWs can not be made
>>    responsive to congestion.  On the other hand, being CBR, they also do
>>    not attempt to capture additional bandwidth when neighboring TCP
>>    flows back off.
>>
>>I suggest adding:
>>
>>    We focus on TDM PWs here as they are the only inelastic PWs that are
>>    likely to be IP-encapsulated in practice.  Most other inelastic PWs
>>    tend to only be used with MPLS.
>>
>>[I'd ask Yaakov and other PW experts to double-check the latter sentence,
>>please.]
>>
>> > A terminology section to roll up all the 
>> acronym expansions (e.g., TDM, CBR)
>> > in one place seems like a good idea.
>>These four acronyms would be good to put into a terminology section:
>>
>>CBR Constant Bit Rate
>>PSN Packet Switched Network
>>PW Pseudowire
>>TDM Time Division Multiplexing
>>
>> >
>> > This text on p.11 could be incendiary:
>> >
>> >       Under this condition it is safe to place the TDM PW along with
>> >       congestion - responsive traffic such as TCP, without causing
>> >       additional congestion.
>> >
>> > First of all to reduce the flammability, "safe" -> "reasonable" and there
>> > are a number of other occurrences of "safe" 
>> that deserve similar treatment ;-
>> > ).
>> > Beyond that, this assertion is dependent on 
>> the operating conditions staying
>> > within that reasonable operating envelope, which entails use of something
>> > like a circuit breaker - I'd say that here, and reference the earlier
>> > circuit breaker discussion.
>>Global substitution: "safe" -> "reasonable", starting with the abstract.
>>
>>Then for this specific text on p.11:
>>
>>OLD
>>         Under this condition it is safe to place the TDM PW along with
>>         congestion - responsive traffic such as TCP, without causing
>>         additional congestion.
>>NEW
>>         Under this condition it is reasonable 
>> to place the TDM PW in competition
>>       with congestion-responsive traffic such 
>> as TCP, as the result should not
>>       produce unacceptable congestion 
>> effects.  That assumes that the operating
>>       conditions remain within the reasonable 
>> operating envelope - as discussed
>>       above, a transport circuit breaker is a 
>> suggested means of managing a TDM
>>       PW so that it is shut down if operating conditions remain persistently
>>       outside the reasonable operating envelope for the PW (e.g., due to a
>>       high packet loss rate).
>
>No need to go there. See my suggestions earlier.
>
>
>> > Figures 7 and 8 could use some shading or lines to indicate the typical
>> > deployed operating regions for the pseudo-wires to visibly show that they
>> > fall under the curves, particularly for larger segment sizes.  I'd also
>> > repeat or refer to the suggestion to use larger
>
>I wanted to suggest this myself, but it is hard 
>to depict fuzzy boundaries. If you can do this, pls do.
>
>Given the PW operating regions are bounded by v 
>low loss probability, the x axis in Figs 7&8 
>ought to be scaled much smaller as well.
>
>Related point: the y axis in Fig 1 ought to max at 10, not 50.
>
>Regards
>
>
>
>Bob
>
>> >
>> > Nits:
>> >
>> > References aren't allowed in abstracts, so remove reference to
>> > [I-D.fairhurst-tsvwg-circuit-breaker]
>> >
>> > Introduction, first paragraph, delete "PWE3" towards end of paragraph -
>> >
>> >    preferably the 4 byte PWE3 control word.
>> >
>> > In addition to its current general statements, the Security Considerations
>> > section should point to the security considerations sections in a few
>> > particularly relevant RFCs - I'd suggest 3985, 4553, 5086 and 5087.
>>After:
>>
>>    This document does not introduce any new congestion-specific
>>    mechanisms and thus does not introduce any new security
>>    considerations above those present for PWs in general.
>>
>>I suggest adding a new paragraph:
>>
>>    For PW security considerations, see the security considerations sections
>>    of the cited RFCS; the security considerations in [RFC3985], [RFC4553],
>>    [RFC5086] and [RFC5087] are particularly applicable to this document.
>>
>> >
>> > Thanks,
>> > --David
>> >
>> >
>> > > -----Original Message-----
>> > > From: pwe3 [mailto:[e-mail]] On Behalf Of Yaakov Stein
>> > > Sent: Thursday, July 24, 2014 4:52 PM
>> > > To: <mailto:[e-mail]>[e-mail]
>> > > Subject: Re: [PWE3] I-D Action: draft-ietf-pwe3-congcons-02.txt
>> > >
>> > > Please look at the PDF version only
>> > > (the txt version does not have the graphs).
>> > >
>> > > Y(J)S
>> > >
>> > >
>> >
>> > _______________________________________________
>> > pwe3 mailing list
>> > <mailto:[e-mail]>[e-mail]
>> > https://www.ietf.org/mailman/listinfo/pwe3
>>
>>_______________________________________________
>>pwe3 mailing list
>><mailto:[e-mail]>[e-mail]
>>https://www.ietf.org/mailman/listinfo/pwe3
>>
>>
>>
>
>________________________________________________________________
>Bob Briscoe,                                                  BT

________________________________________________________________
Bob Briscoe,                                                  BT