Re: [Gen-art] Genart last call review of draft-ietf-rmcat-eval-test-08
Zaheduzzaman Sarker <zaheduzzaman.sarker@ericsson.com> Thu, 07 February 2019 13:27 UTC
Return-Path: <zaheduzzaman.sarker@ericsson.com>
X-Original-To: gen-art@ietfa.amsl.com
Delivered-To: gen-art@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83D941294D0 for <gen-art@ietfa.amsl.com>; Thu, 7 Feb 2019 05:27:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -8.853
X-Spam-Level:
X-Spam-Status: No, score=-8.853 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-4.553, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com header.b=a0+04aBx; dkim=pass (1024-bit key) header.d=ericsson.com header.b=iTTzlsNe
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ASwSlnpNeOaS for <gen-art@ietfa.amsl.com>; Thu, 7 Feb 2019 05:27:50 -0800 (PST)
Received: from sesbmg23.ericsson.net (sesbmg23.ericsson.net [193.180.251.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7C23712D4E8 for <gen-art@ietf.org>; Thu, 7 Feb 2019 05:27:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; d=ericsson.com; s=mailgw201801; c=relaxed/relaxed; q=dns/txt; i=@ericsson.com; t=1549546065; x=1552138065; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:CC:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=zjiq5zhA1XEgWYys9JMW86fESad778MPxiiDYvJjLak=; b=a0+04aBxqQvmrJ9bKuVfnWlGJBkS6mOTTaSUuB4jjyegWwQdeoN8xE1k/InXCRLC HdBNRJgKizfBtxDfeFNg0BkZ7dfGkc9fNS5yG3AqbXgqI62I8BtX01Z1PD5O/rM/ lPPMsbUUZev+geOw7ko5NO1cmuKR7aaDOUj0cLShHnc=;
X-AuditID: c1b4fb25-209009e000005ff7-2b-5c5c3251e372
Received: from ESESBMB502.ericsson.se (Unknown_Domain [153.88.183.115]) by sesbmg23.ericsson.net (Symantec Mail Security) with SMTP id 66.E0.24567.1523C5C5; Thu, 7 Feb 2019 14:27:45 +0100 (CET)
Received: from ESESBMB503.ericsson.se (153.88.183.170) by ESESBMB502.ericsson.se (153.88.183.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3; Thu, 7 Feb 2019 14:27:44 +0100
Received: from EUR02-HE1-obe.outbound.protection.outlook.com (153.88.183.157) by ESESBMB503.ericsson.se (153.88.183.170) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1466.3 via Frontend Transport; Thu, 7 Feb 2019 14:27:44 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zjiq5zhA1XEgWYys9JMW86fESad778MPxiiDYvJjLak=; b=iTTzlsNe1+pvY5iP3r7e04hhieZen/Rw9doSEeLTuwlI+6Z/gETiebLTSv2yVApSzQgQcaNkwpNRWHHq++HomUEtGHuUPBYaGkHWJO5sQBFY84fnGSmnhELOTRPLUncEVgyCAOwZW3Fi0gNLwHcS+tUqOGepKUpQJOAVX1Vj6Z4=
Received: from HE1PR0701MB3020.eurprd07.prod.outlook.com (10.168.93.142) by HE1PR0701MB2170.eurprd07.prod.outlook.com (10.168.36.19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1622.9; Thu, 7 Feb 2019 13:27:43 +0000
Received: from HE1PR0701MB3020.eurprd07.prod.outlook.com ([fe80::f958:b09:cfd7:5878]) by HE1PR0701MB3020.eurprd07.prod.outlook.com ([fe80::f958:b09:cfd7:5878%6]) with mapi id 15.20.1622.012; Thu, 7 Feb 2019 13:27:43 +0000
From: Zaheduzzaman Sarker <zaheduzzaman.sarker@ericsson.com>
To: Stewart Bryant <stewart.bryant@gmail.com>, "gen-art@ietf.org" <gen-art@ietf.org>
CC: "rmcat@ietf.org" <rmcat@ietf.org>, "ietf@ietf.org" <ietf@ietf.org>, "draft-ietf-rmcat-eval-test.all@ietf.org" <draft-ietf-rmcat-eval-test.all@ietf.org>
Thread-Topic: Genart last call review of draft-ietf-rmcat-eval-test-08
Thread-Index: AQHUvL/qZJk7O4BakUOs2Sxp0DvliqXUaREA
Date: Thu, 07 Feb 2019 13:27:42 +0000
Message-ID: <104FE636-0A18-4E3B-B7BD-F2DA3748161B@ericsson.com>
References: <154930852182.28785.5364082865560557648@ietfa.amsl.com>
In-Reply-To: <154930852182.28785.5364082865560557648@ietfa.amsl.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.15.0.190117
authentication-results: spf=none (sender IP is ) smtp.mailfrom=zaheduzzaman.sarker@ericsson.com;
x-originating-ip: [192.176.1.94]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; HE1PR0701MB2170; 6:N3gXQUUANZDQM4YuiI8+AOEnZUE3v0qgpcom8smIZxordJOzXvq1SgDM1QpoN4K5z47geS5zQoaB70SZW2fekMR9NO37ZpfK5FANdJvrs0lxTiyyFPP5CZ4TYBAJ8KuXmCl0wYGhY+K8AGsk2w3008dvNeCwRBj+uq/6g6ZAN7P7wrI8MFIdmoI/hsSHvnNPLTGteBmU/lAPu95vTDvL/aJX17JkfatfmaIiP35gn7lpPC5UzyU7uzOc8qqVEl2JyPlKnC1msYHCorui8K0/i7WYoA0Fb1tt0LgjfiOmdgkLxa79Qws0KS5v1rtwMTQzpCNyJFCLIyAOZBGbTS6oF49yXLd+Tx4WqF9uLz7RJWTu3XXCCl4d09CvNHbhV9dK4rtKTDUxm/RQiQWofwH3snz4CpnD7QDVx40Djk/dlHkoDQ5vbZXizzJis2o6HhmH9DWYSRXAyn1qpr2LKQKNNA==; 5:xQGn3XCwifYkRSZ16tkc+XMX7qbX1o5ls0tZ7Zd0/8ONHuJUuboSCYWE07ZSYVMNfE/o1QjweTST+JyDAlpSelb9m9PLlKzY85MnqLTGP8PrKLxJS7umY6e1PJf7v94BYmV/f6DmsiDuGsueFCbE1WLYL8crw5xVqFCX2dPO0GegnIV932lYiK+/0cWefp2RoqN8sPSBVz3gnvgyPLdwrA==; 7:DTv+pPHxApZPpyK1H33O51U4m8rXzPzr0XuXUFebwpBFm/jXHRjkEbFxPVasabhEw1kjqojjR/12Z9GS+QUxIisgpcWGag7/SOK7MewEHHr9R1Ng/QUTkRgFpQcJyFGLIcmlni35nmV21AjLrv/Xsg==
x-ms-office365-filtering-correlation-id: e0e1052a-4ea8-4c55-5036-08d68d000a3b
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600110)(711020)(4605077)(2017052603328)(7153060)(7193020); SRVR:HE1PR0701MB2170;
x-ms-traffictypediagnostic: HE1PR0701MB2170:
x-microsoft-antispam-prvs: <HE1PR0701MB2170368430944EA78515DC399F680@HE1PR0701MB2170.eurprd07.prod.outlook.com>
x-forefront-prvs: 0941B96580
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(396003)(366004)(346002)(136003)(376002)(189003)(199004)(43544003)(54094003)(99286004)(229853002)(105586002)(476003)(486006)(4326008)(256004)(44832011)(86362001)(33656002)(446003)(966005)(561944003)(71200400001)(97736004)(26005)(53936002)(478600001)(7736002)(14444005)(71190400001)(11346002)(14454004)(2616005)(186003)(305945005)(2501003)(102836004)(36756003)(83716004)(58126008)(316002)(25786009)(66066001)(8676002)(106356001)(76176011)(6506007)(6512007)(6306002)(81156014)(3846002)(81166006)(6486002)(66574012)(6116002)(8936002)(68736007)(6436002)(110136005)(2906002)(54906003)(6246003)(82746002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0701MB2170; H:HE1PR0701MB3020.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: OvFR85DaOYfSnLb1KbcwujvSqYMLMvquvzSpJzhGz08EMEZFNfuSX4lhUBUJ1CiARFu17bRclE6t/7BTfMMnXdAUe5aUaHOOnObQXaX5sWXwQqw36434FawQnADo+e4CN4Ya8g+431gYS8Bs3F4jTV/KaPFLWvdq5ipsCG6KHe9YjuIGvdk3ZkuMjgp4TRGieqGrC1OdowY0hM8l+kzQXf2quNgiXBhGzGivWdz/xBYifHdxpWKDL3nRkPlBBv35iCKh6UjhsjZvDa3j3RMMF/GkIo/M8HJrixfNH7BibtYx6sPw9yIehTO0JJe5bGTO9Xf2Lht2SHniFJnbXb8/Zr09+jIyXPD41Rov0Y9TVW3b7UUdNpSzFkpIboMOewvsx3M7umPCEMnpLKE49uVP6ZF1dE3Kd1jul5lg/+v6Lxw=
Content-Type: text/plain; charset="utf-8"
Content-ID: <B05AE3798A6D6141890FA9CEA35252C3@eurprd07.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: e0e1052a-4ea8-4c55-5036-08d68d000a3b
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Feb 2019 13:27:42.9288 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0701MB2170
X-OriginatorOrg: ericsson.com
X-Brightmail-Tracker: H4sIAAAAAAAAA01Sa0iTURjmfJft0xocl7Y3K8hlRTdvGSzSLpRiSGJY0WWhSz/U8tY+k+Yv MYXc/KFlmqs5p5ORFpqJt4pwWTP9Uaa2nGUXTVmJJaFWaub2Lejfc57bOe/LYUixnvZmktMy WWWaIkUqcKfKT7Rw248EyeUBM307ZRV6LS0b+PKDko3d01OyusHvAln3B8U+OqJN+04YYTT+ IqKJU+4hCWxKchar9N8T555UbOuhMwrjLk09GhDkoJdn1MiNARwMpTljlBq5M2LciaClMA85 BDGeRpB3xZcXqglobxonHAcKF5FQWzHnipQQYOqdckVGEPQOZ6kRwwhwCIxURzloT3wc+hqs pMNPYj2Cbx3NAoewAodBb/MkxZvCoabsqQsHgamsmHBgCvvCzdxFJxbhvdA/cZXm7zoAg3aj s8cNH4RnBbecWYRXwmz3HaefxBKwjeoJfk4MxocvSB57gX3kj7PHC/vDnKnHlY2H69P5At6z DvKtl2ker4VXeg1yDAA4TwjqT59dRYeh1tJF8cIQgk7rNSEvbIOqhRmXKR3s8yZX0xrIqTQJ +YBBAPU6I1mE/LX/vVa7tD0Sb4b6dhcdAV0DQzSPfaBE81GodS7DA56Xj1KViK5FXhzLnU1N DNrhxyqT4zkuPc0vjc1sREvfpqNpbkMr6pvYb0aYQdLlotxNcrmYVmRxqlQzAoaUeop6A5Yo UYJClc0q02OVF1NYzoxWM5RUIpoXe8jFOFGRyZ5n2QxW+U8lGDfvHLRMXKbQDIsWHkS9Dtz6 tkGn2tVJPT42u6o0++5pzYXfeZX3YyyG+TeS8nR2scKg86vxMMc3xZkgNppWNVLvO27brKHF k0pJddh4XXfBz2x5iMq28cZug73VcLLfvj5fZGmLsRwKj4y09SweDfUZ76R0PgIm9mtw1blu SGp7IqW4JEXgFlLJKf4CSMI7QTIDAAA=
Archived-At: <https://mailarchive.ietf.org/arch/msg/gen-art/8ZedrJUyBCzasFCFrftrn_tIeXk>
Subject: Re: [Gen-art] Genart last call review of draft-ietf-rmcat-eval-test-08
X-BeenThere: gen-art@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "GEN-ART: General Area Review Team" <gen-art.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/gen-art>, <mailto:gen-art-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/gen-art/>
List-Post: <mailto:gen-art@ietf.org>
List-Help: <mailto:gen-art-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/gen-art>, <mailto:gen-art-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 07 Feb 2019 13:27:53 -0000
Hi Stewart, Thanks for a good review. For the security consideration section, we can use stronger words if that is required. This document merely specifies test cases when people are testing their algorithm in a controlled environment and does not specify protocol usage. I was wondering if using normative language is an overkill here. For those reasons we are actually thinking of taking out 2119 usage completely. I have a modified text proposal below. Please see inline below for more. BR Zahed On 2019-02-04, 20:29, "Stewart Bryant" <stewart.bryant@gmail.com> wrote: Reviewer: Stewart Bryant Review result: Almost Ready I am the assigned Gen-ART reviewer for this draft. The General Area Review Team (Gen-ART) reviews all IETF documents being processed by the IESG for the IETF Chair. Please treat these comments just like any other last call comments. For more information, please see the FAQ at <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>. Document: draft-ietf-rmcat-eval-test-08 Reviewer: Stewart Bryant Review Date: 2019-02-04 IETF LC End Date: 2019-02-11 IESG Telechat date: Not scheduled for a telechat Summary: A well written documents an close to being ready for publication. I am concerned that the Security section is weak on use outside a controlled environment. There are a fair number of minor issues and nits that need attention, but most of them are simple to fix. One concern that I have that I doubt is readily fixable is that long multi-nested lists do not work well in paginated ASCII with line spaces and sometimes it is difficult to be sure of the context of a test element note. [ZS] I share the concern here, but I don’t think I have a better alternative now. Major issues: 8. Security Considerations The security considerations in [I-D.ietf-rmcat-eval-criteria] and the relevant congestion control algorithms apply. The principles for congestion control are described in [RFC2914], and in particular any new method MUST implement safeguards to avoid congestion collapse of the Internet. The evaluation of the test cases are intended to be run in a controlled lab environment. SB> I wonder if there shouldn't me a MUST in that sentence? SB> There have been issues on SP networks with users running unsuitable SB> performance benchmarks on live networks, including complaints to the SB> operators concerning the results achieved. Hence, the applications, simulators and network nodes ought to be well-behaved and should not impact the desired results. It is important to take appropriate caution to avoid leaking non-responsive traffic from unproven congestion avoidance techniques onto the open Internet. SB> Again I am surprised this is not much stronger in prohibiting SB> use on the Internet. [ZS] what about : " The evaluation of the test cases are intended to be run in a controlled lab environment. Hence, the applications, simulators and network nodes must be well-behaved and should not impact the desired results. Moreover, proper measures must be taken to avoid leaking non-responsive traffic from unproven congestion avoidance techniques onto the open Internet. " ======== Minor issues: This memo describes a set of test cases for evaluating congestion control algorithm proposals for real-time interactive media. SB> It would be useful to add here the statement in the abstract that SB> these tests should be done in a controlled environment. [ZS] The abstract mentions the this "This document describes the test cases to be used in the performance evaluation of such congestion control algorithms in a controlled environment." We can repeat that in the intro as well. =========== Expected behavior: depending on the convergence observed in test case 5.1 and 5.2, the candidate algorithm may be able to avoid congestion collapse. In the worst case, the media stream will fall to the minimum media bit rate. SB> Do you need to specify the variant of TCP? You do state it later, but some comment here would be useful. [ZS] Not sure what would be useful to describe here more, the expected behaviour is not really coupled to what TCP congestion control is used, the general demand is to avoid congestion collapse here. SB> What behaviour do you expect the TCP to show. It would be bad if SB> an aggressive media application kill the TCP completely. [ZS] I don’t think we should say anything about TCP behaviour here. The idea is to test the new congestion control behaviour with available TCP behaviours not vice versa. But TCP can certainly improve its performance from the test results (. ============ the first flow (S1) MUST arrive at a steady-state rate approximately twice of that of the other two flows (S2 and S3). SB> I am not sure what you mean by priority I assume that you mean SB> QoS ranking in the routing system. In which case I don't see SB> how you can expect the result you specify. [ZS] no this is not about routing priority on the network nodes, this is at the media sender. When a media sender has multiple flows that shares the same bottleneck then the media sender can use techniques to distribute the available bandwidth to the multiple flows that it is sending. The point here is the media flows should get their share of the available bandwidth as per the priority set by the application. ============ Expected behavior: the candidate algorithm is expected to achieve full utilization at both bottleneck links without starving any of the three congestion controlled media flows. SB> I am not sure what you mean by this. Do you mean that the bottlenecks SB> will saturate, but make no comment about how much of the bottleneck SB> capacity each flow gets for itself? [ZS] bottlenecks will saturate -- yes. The success criteria --- the existence of multiple bottleneck should not result in flow starvation to any flows that is sharing those bottlenecks. Yes, there is no comment on fairness here explicitly. Not sure we need one here, do we? ============ Nits/editorial comments: [ZS] thanks for those nits. will take care of them. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 9 instances of too long lines in the document, the longest one being 4 characters in excess of 72. == Outdated reference: A later version (-06) exists of draft-ietf-rmcat-wireless-tests-05 =========== 3. Structure of Test cases SB> In the text below it was sometimes hard to get the context right in the SB> triple (or more) nested list. Please consider using subsections or some other SB> demarcation. =========== + Bottleneck queue type: for example, Droptail, FQ-CoDel, or PIE. SB> There need references, and by convention expansion on first use. ========== + Path loss ratio: characterizes the non-congested, additive, losses to be generated on the end-to-end path. MUST SB> s/MUST/This MUST/ ? ========== B. Variation in sending bit rate and goodput. Mainly observing the frequency and magnitude of oscillations. SB> goodput needs a reference or a definition. I don't think it is a universally known term. =========== Expected behavior: the candidate algorithm is expected to detect the path capacity constraint, converges to the bottleneck link's capacity SB> s/converges/converge/ =========== Due to asymmetric nature of the link SB> s/Due to/Due to the/ =========== SB> Is there a diagram error in the figure above? Figure 6: Testbed Topology for TCP vs congestion controlled media Flows =========== have the same bandwidth share on the link. It has to make it's way SB>s/it's/its/ =========== The candidate algorithm MUST reflect the relative priorities assigned to each media flow. In the previous example, SB> An explicit reference to the test would help the reader ==========
- [Gen-art] Genart last call review of draft-ietf-r… Stewart Bryant
- Re: [Gen-art] Genart last call review of draft-ie… Zaheduzzaman Sarker
- Re: [Gen-art] [rmcat] Genart last call review of … Gorry Fairhurst
- Re: [Gen-art] Genart last call review of draft-ie… Stewart Bryant
- Re: [Gen-art] Genart last call review of draft-ie… Mirja Kuehlewind