Re: [rmcat] [Gen-art] Genart last call review of draft-ietf-rmcat-wireless-tests-08

Alissa Cooper <alissa@cooperw.in> Tue, 03 March 2020 20:36 UTC

Return-Path: <alissa@cooperw.in>
X-Original-To: rmcat@ietfa.amsl.com
Delivered-To: rmcat@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 381D43A0913; Tue, 3 Mar 2020 12:36:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cooperw.in header.b=IIiIZkZC; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=IYNwUDW8
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mVplVem0Gyqt; Tue, 3 Mar 2020 12:36:24 -0800 (PST)
Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5D9103A090E; Tue, 3 Mar 2020 12:36:21 -0800 (PST)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 90EA66A4; Tue, 3 Mar 2020 15:36:19 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 03 Mar 2020 15:36:20 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cooperw.in; h= content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=fm2; bh=a Onadmeu6GxMLzlSHAw3C2gsrZGq0RHVWI0eFLS1OEE=; b=IIiIZkZCNLc0aDwRy 1kgBQcaauHBuLet6wlIJCFeuApdX5lE9SoWRKtxrWCLJS02xiWkkIwqG/uozChD+ 7fmfsaaUzkiq+txxjxNyBA8tAjYa8IhonwwdXE6qpoG/vaUm6J6O2mjA7sIXgMPu XY/qKYWdGGywelT7PYEf8BA/pzsaKE1vo3r9XKMjGTRRfDV83rH33xZUhgzo5om6 y39swOufdpZELbyHqA+jvRl4VyhLkywT+4Vi7fEd9yTPZP3edWef+geMA4mr8mgX z0Up+uMa9sRpF3cSDU7SOYmpmmCRhHGbKtSZDeMmIyyHEBg0ZtwmePyYJEze/KkW pqsqQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=aOnadmeu6GxMLzlSHAw3C2gsrZGq0RHVWI0eFLS1O EE=; b=IYNwUDW8JJr+o76yfM2OD2nXPrP8xJqRufzbdBJW5JBk3sZ0sziz+XNIL Eeh5V8CW6zRwT86zxls5t4gILS2SvrAl1wJBOvqDZFyv0rNodmav0xOfanIG4QOy x5qqbQF3oTSDDkHTP3DsOYU2P+NINy3sbkgMQZBXZeRwj+o9HUmnOcDj2d5q2nvj 81cIFKGnd4VQCZ77ZpN7f4rk8UgC4crNqMhL12sFishAlmELq3qPFkP86NYFrguX Oya1Sr8kyMr7Jbul/+prhrUCC+uJUAvGkEzj57v2SbkvWdgO2+/DTzNNPJaNKI62 TL5SWKhCYrl5FbyjkpFUjMAmkI1Ow==
X-ME-Sender: <xms:wb9eXgqmlA--8hsS1qHODuBdL9Jag0F0YCvpgbSz7MrnxO8WHYvJ5Q>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedruddtiedgudegvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpegtggfuhfgjfffgkfhfvffosehtqhhmtdhhtdejnecuhfhrohhmpeetlhhi shhsrgcuvehoohhpvghruceorghlihhsshgrsegtohhophgvrhifrdhinheqnecuffhomh grihhnpehivghtfhdrohhrghenucfkphepudejfedrfeekrdduudejrdektdenucevlhhu shhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegrlhhishhsrgestg hoohhpvghrfidrihhn
X-ME-Proxy: <xmx:wb9eXoPrXR9G-33cj322GF39Kl6QqiuPT2wbrVp_8_v9avsANS5eXw> <xmx:wb9eXkeP47nKCLLlxpywx1oCFkDuctPtop7od7FxJR7MJYlaqScEaQ> <xmx:wb9eXuWcEGvzTBpI25F1p_6aYNegY5dFRzNIcDQEr8SO-CL8kf2kXA> <xmx:w79eXqBhAO_YjNXkpFh7qqt3gvbTsJ2x3y64awFNh3U-pqKAlk6kxQ>
Received: from rtp-alcoop-nitro2.cisco.com (unknown [173.38.117.80]) by mail.messagingengine.com (Postfix) with ESMTPA id 7ACF73060F09; Tue, 3 Mar 2020 15:36:17 -0500 (EST)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Alissa Cooper <alissa@cooperw.in>
In-Reply-To: <D86FE41A-2260-47C3-97F9-B72ACC1C5886@cisco.com>
Date: Tue, 3 Mar 2020 15:36:15 -0500
Cc: "gen-art@ietf.org" <gen-art@ietf.org>, "last-call@ietf.org" <last-call@ietf.org>, "draft-ietf-rmcat-wireless-tests.all@ietf.org" <draft-ietf-rmcat-wireless-tests.all@ietf.org>, "rmcat@ietf.org" <rmcat@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <B4C9B7CE-6FE0-4154-97F2-864991991F50@cooperw.in>
References: <157965034086.28971.6105777873164080956@ietfa.amsl.com> <D86FE41A-2260-47C3-97F9-B72ACC1C5886@cisco.com>
To: "Xiaoqing Zhu (xiaoqzhu)" <xiaoqzhu=40cisco.com@dmarc.ietf.org>, Gyan Mishra <hayabusagsm@gmail.com>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/rmcat/R-gc3rGT_ouwAqmDJCDdl4PKbN4>
Subject: Re: [rmcat] [Gen-art] Genart last call review of draft-ietf-rmcat-wireless-tests-08
X-BeenThere: rmcat@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." <rmcat.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rmcat>, <mailto:rmcat-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rmcat/>
List-Post: <mailto:rmcat@ietf.org>
List-Help: <mailto:rmcat-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rmcat>, <mailto:rmcat-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Mar 2020 20:36:26 -0000

Gyan, thanks for your review. Ziaoqing, thanks for your responses. I entered a No Objection ballot.

Alissa


> On Feb 27, 2020, at 12:36 PM, Xiaoqing Zhu (xiaoqzhu) <xiaoqzhu=40cisco.com@dmarc.ietf.org> wrote:
> 
> Hi Gyan, 
> 
> Thanks a lot for your very thorough review of this draft, and for your valuable comments. We have uploaded the revised version (-09) to address some of the concerns and suggestions you've raised. 
> 
> Please also see our replies below inline, tagged [authors].  Let us know if you have any further thoughts or comments __ 
> 
> Best Regards,
> Xiaoqing, on behalf of all authors.
> 
> 
> On 1/21/20, 3:46 PM, "Gyan Mishra via Datatracker" <noreply@ietf.org> wrote:
> 
>    Reviewer: Gyan Mishra
>    Review result: Ready with Issues
> 
>    I am the assigned Gen-ART reviewer for this draft. The General Area
>    Review Team (Gen-ART) reviews all IETF documents being processed
>    by the IESG for the IETF Chair.  Please treat these comments just
>    like any other last call comments.
> 
>    For more information, please see the FAQ at
> 
>    <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
> 
>    Reviewer: Gyan Mishra
>    Review result: Ready with Minor Issues
> 
>    Document: draft-ietf-rmcat-wireless-tests-08
>    Reviewer: Gyan Mishra
>    Review Date: 2020-01-17
>    IETF LC End Date: 2020-01-21
>    IESG Telechat date: Not scheduled for a telechat
> 
>    Summary: Ready, but with nits and minor issues that should be addressed.
> 
>    Major issues: None
> 
>    Minor issues:  This document describes test cases for evaluating performance of
>    RTP congestion control algorithms over LTE & WIFI networks.
> 
>    Section 1 It is mentioned a number of instance where “wired” is mentioned where
>    I believe “wireless” or “WIFI” is what is meant to be stated.  Please check.
> 
> [authors] Thanks for raising this concern. We’ve doubled checked on the usage of “wired” in all three instances in the introduction section, and believe it is intended as such.  Our intention was to contrast the characteristics of wireless networks from those of wired networks, as a motivation of the need for this draft.   We have, however, taken the opportunity to revise those sentences to make them less wordy.
> 
>    I would recommend  to use WIFI to refer to local LAN WIFI and use cellular to
>    refer to a mobile handset, and not use the term “wireless” as that could be
>    confusing.
> 
> [authors] We agree with the suggestion that it is preferable to refer specifically to WiFi or cellular whenever possible and have mostly edited out the word “wireless”. In a few places, however, it is rather cumbersome to always say “cellular and WiFi networks” when our intention is to simply contrast the difference between wireless and wired communication. We have therefore stuck with the term “wireless” in those cases.
> 
>    I would recommend not using LTE to refer to Cellular from a general mobile
>    handset point of view, since LTE refers to 4G and Cellular could be any -
>    2G,3G,4G,5G etc
> 
> [authors]  We agree with this proposed change. The text has been updated accordingly with a few exceptions: a) mentioning LTE/5G as an example of operational cellular network; b) pointing to the LTE simulator in NS-3; and c) referencing the corresponding 3GPP standard documents.
> 
>    Section 3 This section also mentioned “wired” where I think the goal of the
>    document is test cases of WIFI & Cellular and adding in wired does confuse the
>    reader as we are talking about quality degradation issues with wireless
>    technologies - WIFI & Cellular.  Please check the verbiage.
> 
> [authors] Thanks for pointing this out. We have checked all three previous usage of the word in this section and have eliminated the mention of it in the introductory discussions. The remaining two uses, however, are needed for describing the proposed test case topology so we left them as is.
> 
>    Section 3 In this sentence you mention 3G support of high bandwidth.  My
>    experience with 3G has not been very slow page loading and that multimedia
>    would suffer.
> 
> [authors] This is a fair complaint. To avoid controversy, we have revised the sentence to now state: “… it is evident that only the more recent radio technologies can support the high bandwidth requirements  …”
> 
> 
>    Section 3 Bottom of page 5 it is mentioned that the combination of multiple
>    access technologies such as one user has LTE connection and another has Wi-Fi
>    connection is kept out of the scope of this document.  Please explain why as
>    the reader would believe it would be in scope as that is the main topic.
> 
> [authors]  When some of the users are on LTE and some of the users are on Wi-Fi, it is only interesting to investigate the test case when the bottleneck of the system is on the wired hop.  Whereas the main focus of this draft is to evaluate how congestion control schemes interactive over the wireless interface. We have added the following statement in the draft to explain why it is out of scope:
> 
> 	“It should be noted that the goal of the following test cases is to evaluate the performance
> 	of candidate algorithms over the radio interface of the cellular network. Hence it is assumed
> 	that the radio interface is the bottleneck link between the communicating peers and that the core
> 	network does not introduce any extra congestion along the path. Consequently, this draft has kept
> 	as out of scope the combination of multiple access technologies involving both cellular and Wi-Fi users.”
> 
>    Section 3 Top of page 6 - I am wondering if there is a better explanation of
>    why you need a test simulator due to unpredictable nature or cellular other
>    than underground mines.
> 
> [authors]  The unpredictable nature of the cellular networks makes test results unrepeatable. Therefore our recommendation of carrying out tests using simulators.
> 
>    Section 3.1.1 Here the fixed user is on a wired LAN connected to mobile user. 
>    You mention wireless interface which makes me wonder is the fixed user actually
>    a WIFI user connected to broadband WIFI router wired to the internet. See in
>    quotes which makes me wonder "The fixed user is connected to the Internet via
>    wired connection with sufficiently high bandwidth, for instance, 10 Gbps, so
>    that the system is resource limited on the {wireless?} interface"
> 
> 
> [authors]  Thanks for raising this point of confusion. The proposed test case comprise of both cellular users and fixed users. They share the wired backhaul link with sufficiently high bandwidth.  As such, the bottleneck of the system is over the cellular radio access link.  We have revised the sentence as below to avoid further confusion:
> 
> “… so that the system bottleneck is on the cellular radio access interface … “
> 
> 
>    Section 4.1 Please explain in why the home access link represents a bottleneck
>    due to its bandwidth. It is not obvious as these days Gigabit and above
>    broadband speeds are available at home.
> 
> 
> [authors] We agree that the scenario where the wired hop remains the bottleneck is becoming less common. This subsection describes test cases to reflect DSL-like home Internet access that's less common but still present especially in other parts of the world.   As acknowledged in this section, these test cases are proposed mainly as a sanity check. Over time, we can perhaps get rid of this section completely.
> 
>    Nits/editorial comments:
> 
>    Draft Date should to be updated.
> 
> [authors] Done
> 
>    Section 3.1.2  1-B Antenna- [2D, 3D] should be defined
> 
> [authors]  The cited reference of the 3GPP test plan contains information for the above terminology.  For this round of revision we have not gathered input from authors familiar with the cellular technology terminologies. We will try to update the definition directly in the text in a later revision. 
> 
>    Section 3.1.2 RTT [40, 150] , should define a unit millisecond
> 
> [authors] Done.
>    Section 3.1.2 4. User Intensity, should define what the values in brackets mean
>    and unit of measure.
> 
> [authors]  The cited reference of the 3GPP test plan contains information for the above terminology.  For this round of revision we have not gathered input from authors familiar with the cellular technology terminologies. We will try to update the definition directly in the text in a later revision.
> 
> Section 3.1.2 7.2.a Media direction: Uplink and Downlink,   should define what is meant by Uplink and Downlink 
> 
> [authors] The definition has been provided in the second paragraph of Section 3.1.1 as well as in the illustration in Fig. 1.
> 
> Section 4.2.3 g N= [4, 8,  12, 16, 20], should define what N is and any unit of measure
> 
> [authors] The variables N and M indicate the number of real-time media flows and competing TCP streams, respectively, as illustrated in Fig. 2. We added a sentence in Sec. 4.1.1 to explicitly explain this
> 
>    _______________________________________________
> 
>    Gen-art mailing list
>    Gen-art@ietf.org
>    https://www.ietf.org/mailman/listinfo/gen-art
> 
> 
> 
> 
> _______________________________________________
> Gen-art mailing list
> Gen-art@ietf.org
> https://www.ietf.org/mailman/listinfo/gen-art