From nobody Tue Feb 15 16:34:18 2022 Return-Path: X-Original-To: ippm@ietfa.amsl.com Delivered-To: ippm@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99C533A1173 for ; Tue, 15 Feb 2022 16:34:16 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8FRSdY2riKmf for ; Tue, 15 Feb 2022 16:34:12 -0800 (PST) Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3F8163A1171 for ; Tue, 15 Feb 2022 16:34:12 -0800 (PST) Received: by mail-ed1-x536.google.com with SMTP id b14so1070827ede.9 for ; Tue, 15 Feb 2022 16:34:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+a1VDdrA0c8hienibdx7MJLjPxuSdeR6tUx5/Gwb0+8=; b=lGCqprGSOBmgMsi/SHRliiWqowDZprhriiT5qGW2KwBaFvFWDXIzHSnxuybQYFORHB 50gc229CVS/oxYz9Qnt00ylflxFMen384nQOtUZPjtPw4bJ6QPN2O0vt+/z4OxerXfKa T1t9HhNAZtq9ZVkSQ0jNVOyHluIAjEbFO+reyhv2APHQmuqekytnxWk+9cHXqrUuIuUs rOKEsz/A8nWlcSxh94vVSP0pRUv71hvUEIa/7muTVLjGIRWR5PRSnumCaDuKKZR4R93T VqD2P9cVvdB1DmesY387KTZOrvxf3U/ejWNjw9lx2NxLNrn8100TynZZu8t4rG7ySHAD O8jQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+a1VDdrA0c8hienibdx7MJLjPxuSdeR6tUx5/Gwb0+8=; b=kqBxY4H8dok5rJpIuO/NgT+8+kgoY9ZburrQy1B49naJbl9KFqXUfJjvEHm02cIRcF 6XtulnNu7ue3Do+OgxDKtz85tvgQDgYCOphOtFcvizlZYVvNPSmGkWYNRBqRl1ZZ0lkE DA5wIyktLXQbn9OPL5gvqsYMnGN8P+dWRG2AhnrxVhKl+5csVcKu/3GKcJcB47NZKF5c k5hwHB9ULRbVmV3s1Fj5fwFXjuJPjlBs9Fb3f9xb+3lZsUmhJpr1bTONEvyqCfgmO6O0 TsPlFGYXqUS+nJbVpyiIsdzOpekxRop6ZQhB6hhMzXq5t/Vy6/68PSa6Rt4L2ktHG4m1 K0sQ== X-Gm-Message-State: AOAM532tVM912xe58JaPQLQzg4+MMajrWmq6GPp//DxBj2Ep26vqW7wU scb02dIb06JGEkFjy8C1b2WoxRQXvUXYygwANLLm+M7l X-Google-Smtp-Source: ABdhPJzTDDyOIOFI0r/oaDjgiDJR3BjTXOtMwLImuVgWr0boBDLzxnkolVKFmF9CIWZXciWIsRzIRlrZPMTD8SXF4so= X-Received: by 2002:aa7:c3d5:0:b0:40f:b885:8051 with SMTP id l21-20020aa7c3d5000000b0040fb8858051mr356023edr.395.1644971649926; Tue, 15 Feb 2022 16:34:09 -0800 (PST) MIME-Version: 1.0 References: <3DC3F6B6-229E-46C6-BD84-2A6A7FE6DD48@apple.com> <9340CFDA-079C-4490-A01C-EB863D365F8F@apple.com> In-Reply-To: From: Greg Mirsky Date: Tue, 15 Feb 2022 16:33:58 -0800 Message-ID: To: Christoph Paasch Cc: Marcus Ihlar , IETF IPPM WG Content-Type: multipart/alternative; boundary="000000000000dc078205d817ce5f" Archived-At: Subject: Re: [ippm] Adoption call for draft-cpaasch-ippm-responsiveness X-BeenThere: ippm@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: IETF IP Performance Metrics Working Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Feb 2022 00:34:17 -0000 --000000000000dc078205d817ce5f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Christoph, thank you for putting your thought into my comments. Your understanding is absolutely correct and the new section, as you've outlined it, would make the document even more useful to a reader. With that plan in place, I support the adoption of the draft and would help with review and comments. Regards, Greg On Tue, Feb 15, 2022 at 4:11 PM Christoph Paasch wrote: > Hello Greg, > > On Feb 8, 2022, at 1:10 PM, Greg Mirsky wrote: > > Hi Christoph, > apologies for the belated response and thank you for sharing interesting > details of you using the measurement method. I think that if the > measurement method can not only provide the Round-trip Per Minute (RPM) > metric but expose the network propagation and residential components of t= he > round-trip delay, then it seems to me, the scope of the draft to be align= ed > with the charter of the IPPM WG and I'll be in favor of the WG adoption o= f > the work. > What do you think? What is the opinion of the authors and the WG? > > > I am assuming that with "residential components" you mean the > server/client-side contribution to the measured latency, right? > > In that case, yes the method does allow to separate these, as > latency-probes are sent on both the load-generating connections and on > separate connections. The difference between the two represents the > "server-side contribution" to the latency. > > I think what would be helpful would be a section in the draft that > explains the different sources of latency (network, server, client) and h= ow > they affect the final RPM-number and how one can separate out these two > components. It is also important to understand that the results are highl= y > implementation-dependent. And explaining that in this section should help= , > I believe. > > Would that be in line with what you are looking for? > > > Thanks, > Christoph > > > > Regards, > Greg > > On Thu, Jan 6, 2022 at 4:42 PM Christoph Paasch wrote= : > >> Hello Greg, >> >> On Jan 6, 2022, at 3:00 PM, Greg Mirsky wrote: >> >> Hi Christoph, >> a happy and healthy New Year to you and All! >> >> >> Happy New Year to you as well! >> >> Thank you for your kind consideration of my notes and detailed responses= . >> Please find my follow-up notes in-line below under the GIM>> tag. >> >> >> Thanks for your replies. Please see inline: >> >> On Wed, Jan 5, 2022 at 10:52 AM Christoph Paasch >> wrote: >> >>> Hello Greg, >>> >>> thanks for your comments. Please see inline: >>> >>> On Dec 22, 2021, at 11:43 PM, Greg Mirsky wrote= : >>> >>> Dear Marcus, Authors, et al, >>> apologies for the belated response. >>> I've read the draft and have some comments to share with you: >>> >>> - as I understand it, the proposed new responsiveness metric is >>> viewed as the single indicator of a bufferbloat condition in a netwo= rk. As >>> I recall, the discussion at Measuring Network Quality for End-Users >>> workshop and on the mailing list >>> >>> indicated, that there=E2=80=99s no consensus on what behaviors, symp= toms can >>> reliably signal the bufferbloat. >>> >>> We are not trying for this responsiveness metric to be "the single >>> indicator of bufferbloat". Bufferbloat can be measured in many differen= t >>> number of ways. And each of these will produce a correct, but a differe= nt >>> result. Thus, "bufferbloat" is whatever the methodology tries to detect= . >>> >>> Let me give an example of two methodologies that are both correct but >>> both will produce entirely different numbers : >>> >>> If we would decide to generate the load by flooding the network with UD= P >>> traffic from a specific 4-tuple and measure latency with parallel ICMP >>> pings. Then, on a over-buffered FIFO queue we would measure huge latenc= ies >>> (thus correctly expose bufferbloat), while on a FQ-codel queue we would= not >>> measure any bufferbloat. >>> >>> If on the other hand, the load-generating traffic is changing the >>> source-port for every single UDP-packet, then in both the FIFO-queue an= d >>> the FQ-codel queue we will measure huge amounts of bufferbloat. >>> >>> Thus, these two methods both produced correct results but with hugely >>> different numbers in the FQ-codel case. [1] >>> >>> Now, while both methods measure some variant of bufferbloat, they both >>> don't measure a realistic usage of the network. >>> >> GIM>> Thank you for the insights. It seems to me that what the method ca= n >> demonstrate is rather the level of efficiency of the AQM in the network = for >> a particular class of applications. >> >> >> Yes, that is a good description. It is for a "particular class of >> applications" and we are trying to make this class of applications >> representative of a "typical user-scenario". (admittedly, we can debate >> forever on what kind of applications are representative and I would love= to >> have that debate :-)). >> >> On the point of "efficiency of the AQM". I would go even further that >> it's not only AQM but also the client- and server-side implementations o= f >> these applications (as noted further below). >> >> >> >>> >>> That is why the "Responsiveness under working conditions" tries to >>> clearly specify how the load is generated and how the latency is being >>> measured. And it does not measure "bufferbloat" but it measures >>> "responsiveness under working conditions" based on the methodology that= is >>> being used (using HTTP/2 or HTTP/3, multiple flows, ...). It does expos= e >>> bufferbloat which can happen in the network. It also exposes certain >>> server-side behaviors that can cause (huge amounts of) additional laten= cy - >>> those behaviors are typically not called "bufferbloat". >>> >> GIM>> Thank you for pointing out that the result of the RTT measurement >> has two contributing factors - network and server. >> >> >> Yes, servers contribute as do the client-side implementations. It's all >> three (client, network, server) that need to work "correctly" to achieve >> good responsiveness. Btw., as we are now gathering more experience with = our >> methodology in different environments we find that the biggest portions = of >> latency actually come from the server-side. We see several seconds of >> latency introduced by the HTTP/2 and TCP implementations. >> >> It seems worth enhancing the method to localize each contribution and >> measure them separately. >> >> >> With the latency measuring probes being sent on load-bearing connections >> and separate connections and with the separate connections serving to >> measure DNS/TCP/... individually, the different data-points actually all= ow >> to localize to some extend. >> >> However, I would be reluctant to dive too deep into >> localization/trouble-shooting/debugging of networks as part of this I-D.= As >> this opens a whole new can of worms. We could then start thinking about >> sending latency-probes while playing with the IP TTL to find which route= r >> is introducing the latency,... It's an entirely different research-topic >> IMO :-) Dave Taht was thinking of starting something along these lines ( >> https://github.com/dtaht/wtbb). >> >> >>> >>> >>> - It seems that it would be reasonable to first define what is being >>> measured, characterized by the responsiveness metric. Having a docum= ent >>> that discusses and defines the bufferbloat would be great. >>> >>> I agree that there is a lack of definition for what "bufferbloat" reall= y >>> is. >>> >>> The way we look at "responsiveness under working conditions" is that it >>> measures the latency in conditions that may realistically happen in >>> worst-case scenarios with end-users/implementations that are non-malici= ous >>> (non-malicious to exclude the UDP-flooding scenario). >>> >>> Thus, I assume we should make a better job at explaining this. The lack >>> of a formal definition of "bufferbloat" doesn't help and thus we are in= deed >>> using this term a bit freely in the current draft. We will improve the >>> Introduction to better set the stage ( >>> https://github.com/network-quality/draft-cpaasch-ippm-responsiveness/is= sues/31 >>> ). >>> >>> >>> - It seems like in the foundation of the methodology described in >>> the draft lies the assumption that without adding new flows the >>> available bandwidth is constant, does not change. While that is most= ly the >>> case, there are technologies that behave differently and may change >>> bandwidth because of the outside conditions. Some of these behaviors= of >>> links with variable discrete bandwidth are discussed in, for example= , RFC >>> 8330 and RFC 8625 >>> . >>> >>> I'm not sure I entirely understand your comment. But let me explain why >>> we are gradually adding new flows: >>> >>> 1. TCP-implementations have usually a fixed limit for the upper bound o= f >>> the receive window. In some networks that upper bound is lower than the= BDP >>> of the network. Thus, the only way to reach full capacity is by having >>> multiple flows. >>> 2. Having multiple connections allows to quicker achieve full capacity >>> in high-RTT networks and thus speeds up the test-duration. >>> 3. In some networks with "random" packet-loss, congestion-control may >>> come in the way of achieving full capacity. Again, multiple flows will = work >>> around that. >>> >> GIM>> I might have asked several questions at once. Let me clarify what = I >> am looking for: >> >> - As I understand the method of creating the "working conditions in a >> network" is based on certain assumptions. First, seems is that the >> bandwidth is symmetrical between the measurement points. Second, that= the >> bandwidth doesn't change for the duration of the measurement session. >> AFAIK, in the access networks, both are not necessarily always the ca= se. >> >> We don't have the assumption that bandwidth is symmetrical (assuming, yo= u >> mean uplink/downlink symmetry - please clarify otherwise). >> >> The load-generating algorithm runs independently for uplink and downlink >> traffic. And it is perfectly fine when both have huge asymmetry. >> >> >> Regarding the stability of the bandwidth: >> You are making a good point indeed that we assume that the bandwidth is >> to some extend stable while ramping up the flows to "working conditions"= . >> Admittedly that assumption does not always hold, and that is one of the >> reasons why we try hard for the test to not take too long. >> I'm not sure how we could adjust the algorithm for varying bandwidth >> without introducing too much complexity. I'm open for suggestions :-) >> >> >> - On the other hand, I might have missed how the method of creating >> the "working conditions" guarantees a symmetrical load between the >> measurement points. >> >> As mentioned above, we don't assume a symmetrical load. Can you show us >> where in the draft we give that impression, so we can fix that? >> >> >>> >>> - Then, I find the motivation not to use time units to express the >>> responsiveness metric not convincing: >>> >>> "Latency" is a poor measure of responsiveness, since it can be hard >>> for the general public to understand. The units are unfamiliar >>> ("what is a millisecond?") and counterintuitive ("100 msec - that >>> sounds good - it's only a tenth of a second!"). >>> >>> >>> Can you expand on what exactly is not convincing to you? Do you think >>> that people will mis-understand the metric or that milli-seconds is the >>> right way to communicate responsiveness to the general public? >>> >> GIM>> Let me try. We know packet delay requirements for AR, VR >> applications. I believe that gamers are familiar with these numbers too. >> The same is likely the case for the industrial automation use cases serv= ed, >> for example, by Deterministic Networking. >> >> >> I can understand that for a technical audience, milli-seconds is easy an= d >> familiar. A non-technical audience might be more open to accepting a new >> "higher-is-better" metric. Responsiveness is something new and abstract = so, >> it's kind of natural that it comes with a new unit. >> >> But I fully recognize that that's a controversial topic and can be >> discussed at length :) >> >> >> Cheers, >> Christoph >> >> >>> >>> Thanks a lot, >>> Christoph >>> >>> [1] And there are many networks that prioritize ICMP pings, thus we >>> could observe even more different results based on what protocol is use= d to >>> measure the latency. >>> >>> >>> On Mon, Dec 6, 2021 at 7:53 AM Marcus Ihlar >> 40ericsson.com@dmarc.ietf.org> wrote: >>> >>>> Hi IPPM, >>>> >>>> >>>> >>>> This email starts an adoption call for >>>> draft-cpaasch-ippm-responsiveness, "Responsiveness under Working >>>> Conditions=E2=80=9D. This document specifies the =E2=80=9CRPM Test=E2= =80=9D for measuring user >>>> experience when the network is fully loaded. The intended status of th= e >>>> document is Experimental. >>>> >>>> >>>> >>>> https://datatracker.ietf.org/doc/draft-cpaasch-ippm-responsiveness/ >>>> >>>> >>>> https://datatracker.ietf.org/doc/html/draft-cpaasch-ippm-responsivenes= s-01 >>>> >>>> >>>> >>>> This adoption call will last until *Monday, December 20*. Please >>>> review the document, and reply to this email thread to indicate if you >>>> think IPPM should adopt this document. >>>> >>>> >>>> >>>> BR, >>>> >>>> Marcus >>>> >>>> >>>> _______________________________________________ >>>> ippm mailing list >>>> ippm@ietf.org >>>> https://www.ietf.org/mailman/listinfo/ippm >>>> >>> _______________________________________________ >>> ippm mailing list >>> ippm@ietf.org >>> https://www.ietf.org/mailman/listinfo/ippm >>> >>> >>> >> > --000000000000dc078205d817ce5f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi=C2=A0Christoph,
thank you for putting your thought = into my comments. Your understanding is absolutely correct and the new sect= ion, as you've outlined it, would make the document even more useful to= a reader. With that plan in place, I support the adoption of the draft and= would help with review and comments.

Regards,
Greg

On Tue, Feb 15, 2022 at 4:11 PM Christoph Paasch <cpaasch@apple.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-l= eft:1px solid rgb(204,204,204);padding-left:1ex">
Hello Greg,

On F= eb 8, 2022, at 1:10 PM, Greg Mirsky <gregimirsky@gmail.com> wrote:

=
Hi Christoph,
apologies for the belat= ed response and thank you for sharing interesting details of you using the = measurement method. I think that if the measurement method can not only pro= vide the Round-trip=C2=A0Per Minute (RPM) metric but expose the network pro= pagation and residential components of the round-trip delay, then it seems = to me, the scope of the draft to be aligned with the charter of the IPPM WG= and I'll be in favor of the WG adoption of the work.
What=C2= =A0do you think? What is the opinion of the authors and the WG?
=

I am assuming that with "= residential components" you mean the server/client-side contribution t= o the measured latency, right?

In that case, yes t= he method does allow to separate these, as latency-probes are sent on both = the load-generating connections and on separate connections. The difference= between the two represents the "server-side contribution" to the= latency.

I think what would be helpful would be a= section in the draft that explains the different sources of latency (netwo= rk, server, client) and how they affect the final RPM-number and how one ca= n separate out these two components. It is also important to understand tha= t the results are highly implementation-dependent. And explaining that in t= his section should help, I believe.

Would that be = in line with what you are looking for?


<= div>Thanks,
Christoph



Regards,<= /div>
Greg

On Thu, Jan 6, 2022 at 4:42 PM Christoph Paasch <cpaasch@apple.com&g= t; wrote:
H= ello Greg,

On Jan 6, 2022, at 3:= 00 PM, Greg Mirsky <gregimirsky@gmail.com> wrote:

=
Hi Christoph,
a happy and healthy New= Year to you and All!

Happy New Year to you as well!

=
Thank you for = your kind consideration of my notes and detailed responses. Please find my = follow-up notes=C2=A0in-line below under the GIM>> tag.

Thanks for your replies. P= lease see inline:

On Wed, Jan 5, 2022 at 10:52 AM Christoph Paasch <cpaasch@apple.com>= ; wrote:
He= llo Greg,

thanks for your comments. Please see inline:

On Dec 22, 2021, at 11:43 PM, Gre= g Mirsky <gre= gimirsky@gmail.com> wrote:

Dear Marcus, Authors, et al,apologies for the belated response.
I've read the draft and = have some comments to share with you:
  • as I understand it,= the proposed new responsiveness metric is viewed as the single indicator o= f a bufferbloat condition in a network. As I recall,=C2=A0the discussion at= Measuring Network Quality for End-Users workshop and on the = mailing list indicated, that there=E2=80=99s no consensus on what behav= iors, symptoms can reliably signal the bufferbloat.
<= /div>
We are not trying for this respons= iveness metric to be "the single indicator of bufferbloat". Buffe= rbloat can be measured in many different number of ways. And each of these = will produce a correct, but a different result. Thus, "bufferbloat&quo= t; is whatever the methodology tries to detect.

Le= t me give an example of two methodologies that are both correct but both wi= ll produce entirely different numbers :

If we woul= d decide to generate the load by flooding the network with UDP traffic from= a specific 4-tuple and measure latency with parallel ICMP pings. Then, on = a over-buffered FIFO queue we would measure huge latencies (thus correctly = expose bufferbloat), while on a FQ-codel queue we would not measure any buf= ferbloat.

If on the other hand, the load-generatin= g traffic is changing the source-port for every single UDP-packet, then in = both the FIFO-queue and the FQ-codel queue we will measure huge amounts of = bufferbloat.

Thus, these two methods both produced= correct results but with hugely different numbers in the FQ-codel case. [1= ]

Now, while both methods measure some variant of = bufferbloat, they both don't measure a realistic usage of the network.<= /div>
GIM>> Thank you for the insi= ghts. It seems to me that what the method can demonstrate is rather the lev= el of efficiency of the AQM in the network for a particular class of applic= ations.

Yes, = that is a good description. It is for a "particular class of applicati= ons" and we are trying to make this class of applications representati= ve of a "typical user-scenario". (admittedly, we can debate forev= er on what kind of applications are representative and I would love to have= that debate :-)).

On the point of "efficienc= y of the AQM". I would go even further that it's not only AQM but = also the client- and server-side implementations of these applications (as = noted further below).



That is why the "Responsiveness under working conditions&= quot; tries to clearly specify how the load is generated and how the latenc= y is being measured. And it does not measure "bufferbloat" but it= measures "responsiveness under working conditions" based on the = methodology that is being used (using HTTP/2 or HTTP/3, multiple flows, ...= ). It does expose bufferbloat which can happen in the network. It also expo= ses certain server-side behaviors that can cause (huge amounts of) addition= al latency - those behaviors are typically not called "bufferbloat&quo= t;.
GIM>> Thank you for poin= ting out that the result of the RTT measurement has two contributing factor= s - network and server.

=
Yes, servers contribute as do the client-side implementations. I= t's all three (client, network, server) that need to work "correct= ly" to achieve good responsiveness. Btw., as we are now gathering more= experience with our methodology in different environments we find that the= biggest portions of latency actually come from the server-side. We see sev= eral seconds of latency introduced by the HTTP/2 and TCP implementations.
It seems worth enhancing the method to locali= ze each contribution and measure them separately.

With the latency measuring probes bein= g sent on load-bearing connections and separate connections and with the se= parate connections serving to measure DNS/TCP/... individually, the differe= nt data-points actually allow to localize to some extend.

However, I would be reluctant to dive too deep into localization/tr= ouble-shooting/debugging of networks as part of this I-D. As this opens a w= hole new can of worms. We could then start thinking about sending latency-p= robes while playing with the IP TTL to find which router is introducing the= latency,... It's an entirely different research-topic IMO :-) Dave Tah= t was thinking of starting something along these lines (https://github.com/dtaht/wtbb)= .

    • It seems that it = would be reasonable to first define what is being measured, characterized= =C2=A0by the responsiveness metric. Having a document that discusses and de= fines the bufferbloat would be great.
    I agree that there is a lack of definition for wh= at "bufferbloat" really is.

    The way we l= ook at "responsiveness under working conditions" is that it measu= res the latency in conditions that may realistically happen in worst-case s= cenarios with end-users/implementations that are non-malicious (non-malicio= us to exclude the UDP-flooding scenario).

    Thus, I = assume we should make a better job at explaining this. The lack of a formal= definition of "bufferbloat" doesn't help and thus we are ind= eed using this term a bit freely in the current draft. We will improve the = Introduction to better set the stage (htt= ps://github.com/network-quality/draft-cpaasch-ippm-responsiveness/issues/31= ).

    • It seems l= ike in the foundation of the methodology described in the draft lies the as= sumption=C2=A0that without adding new flows the available=C2=A0bandwidth=C2= =A0is constant, does not change. While that is mostly the case, there are t= echnologies that behave differently and may change bandwidth because of the= outside conditions. Some of these behaviors of links with variable discret= e=C2=A0bandwidth are discussed in, for example,=C2=A0RFC 83= 30=C2=A0and RFC 8625.
    =
    I'm not sure I entirely understand = your comment. But let me explain why we are gradually adding new flows:

    1. TCP-implementations have usually a fixed limit for= the upper bound of the receive window. In some networks that upper bound i= s lower than the BDP of the network. Thus, the only way to reach full capac= ity is by having multiple flows.
    2. Having multiple connections a= llows to quicker achieve full capacity in high-RTT networks and thus speeds= up the test-duration.
    3. In some networks with "random"= ; packet-loss, congestion-control may come in the way of achieving full cap= acity. Again, multiple flows will work around that.
    =
    GIM>> I might have asked several questions at once.= Let me clarify what I am looking for:
    • As I understand th= e method of creating the "working conditions in a network" is bas= ed on certain assumptions. First, seems is that the bandwidth is symmetrica= l between the measurement points. Second, that the bandwidth doesn't ch= ange for the duration of the measurement session. AFAIK, in the access netw= orks, both are not necessarily always the case.
    =
    We don't have the assumption that bandwid= th is symmetrical (assuming, you mean uplink/downlink symmetry - please cla= rify otherwise).

    The load-generating algorithm run= s independently for uplink and downlink traffic. And it is perfectly fine w= hen both have huge asymmetry.


    Regar= ding the stability of the bandwidth:
    You are making a good point = indeed that we assume that the bandwidth is to some extend stable while ram= ping up the flows to "working conditions". Admittedly that assump= tion does not always hold, and that is one of the reasons why we try hard f= or the test to not take too long.
    I'm not sure how we could a= djust the algorithm for varying bandwidth without introducing too much comp= lexity. I'm open for suggestions :-)
      On the other hand, I might have missed how the method of creating the &quo= t;working conditions" guarantees a symmetrical load between the measur= ement points.
    As m= entioned above, we don't assume a symmetrical load. Can you show us whe= re in the draft we give that impression, so we can fix that?

    =

    • Then, I find the motivation not to use time un= its to express the responsiveness metric not convincing:
    <= div>

    Thanks a lot,
    Christoph

    [1] And there are many networks that prioritize ICMP ping= s, thus we could observe even more different results based on what protocol= is used to measure the latency.


    On Mon, Dec 6, 2021 at 7:53 AM Marcus Ihlar <marcus.ihlar=3D40ericsson.com@dmarc.ietf.org> wrote:

    Hi IPPM,<= u>

    <= /u>=C2=A0

    This email starts an adoption call for draft-cpaasch-ippm-responsiven= ess, "Responsiveness under Working Conditions=E2=80=9D. This document = specifies the =E2=80=9CRPM Test=E2=80=9D for measuring user experience when= the network is fully loaded. The intended status of the document is Experiment= al. =C2=A0=C2=A0

    =C2=A0

    <= span style=3D"font-size:12pt">htt= ps://datatracker.ietf.org/doc/draft-cpaasch-ippm-responsiveness/=

    https://datatracker.ietf.org/= doc/html/draft-cpaasch-ippm-responsiveness-01

    =C2=A0

    This adoption call will last until Monday, December 20. Please review the document, and reply to this e= mail thread to indicate if you think IPPM should adopt this document.

    =C2=A0

    BR,

    Marcus

    =C2=A0

    _______________________________________________
    ippm mailing list
    ippm@= ietf.org
    https://www.ietf.org/mailman/listinfo/ippm<= br>
    _______________________________________________
    ippm mailing list
    ippm@ietf.org
    https://= www.ietf.org/mailman/listinfo/ippm



    --000000000000dc078205d817ce5f--