Re: [rmcat] AD review of draft-ietf-rmcat-eval-criteria-10

Mirja Kuehlewind <ietf@kuehlewind.net> Wed, 08 January 2020 13:00 UTC

Return-Path: <ietf@kuehlewind.net>
X-Original-To: rmcat@ietfa.amsl.com
Delivered-To: rmcat@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B82161200DB; Wed, 8 Jan 2020 05:00:39 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PauCfvGdXa7h; Wed, 8 Jan 2020 05:00:37 -0800 (PST)
Received: from wp513.webpack.hosteurope.de (wp513.webpack.hosteurope.de [IPv6:2a01:488:42:1000:50ed:8223::]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0FAF912011C; Wed, 8 Jan 2020 05:00:37 -0800 (PST)
Received: from 200116b82ce1a4005104981634920ee5.dip.versatel-1u1.de ([2001:16b8:2ce1:a400:5104:9816:3492:ee5]); authenticated by wp513.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) id 1ipAwr-0003nk-ED; Wed, 08 Jan 2020 14:00:33 +0100
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Mirja Kuehlewind <ietf@kuehlewind.net>
In-Reply-To: <3B17A1D8-6743-4513-853B-F8FEAFE604DC@kuehlewind.net>
Date: Wed, 8 Jan 2020 14:00:32 +0100
Cc: rmcat@ietf.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <EA9CBD03-F210-4FBE-9646-3CBCB2B25ACF@kuehlewind.net>
References: <3B17A1D8-6743-4513-853B-F8FEAFE604DC@kuehlewind.net>
To: draft-ietf-rmcat-eval-criteria.all@ietf.org
X-Mailer: Apple Mail (2.3445.104.11)
X-bounce-key: webpack.hosteurope.de;ietf@kuehlewind.net;1578488437;ed6292ae;
X-HE-SMSGID: 1ipAwr-0003nk-ED
Archived-At: <https://mailarchive.ietf.org/arch/msg/rmcat/eFtTC-jvFrbCNULLcxD1onIjRd8>
Subject: Re: [rmcat] AD review of draft-ietf-rmcat-eval-criteria-10
X-BeenThere: rmcat@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." <rmcat.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rmcat>, <mailto:rmcat-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rmcat/>
List-Post: <mailto:rmcat@ietf.org>
List-Help: <mailto:rmcat-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rmcat>, <mailto:rmcat-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Jan 2020 13:00:40 -0000

Hi again,

Actually one more question: There is appendix A which reads like there is still work to do. Maybe that is information that should be rephrased and added in the intro?

Further the terminology section (2) references to a bunch of RTP document normatively. However, I don’t think this terminology is actually used in this draft. Can you please double-check and adapt the references accordingly!?

And again about the spilt between normative and informational. I think this document does not need to have the test RFCs as normative references (but only the other way around: test RFCs reference this doc normatively). However, given section 6.2., RFC8593 should probably be a normative reference.

And as I’m looking at the references: I think section 3 could probably reference more closely some sections of RFC5166.

Mirja



> On 8. Jan 2020, at 13:17, Mirja Kuehlewind <ietf@kuehlewind.net> wrote:
> 
> Hi authors,
> 
> I have reviewed draft-ietf-rmcat-eval-criteria-10 in order to move it ahead. I think it’s really for IETF last call and we can start it right now if you what. 
> Of course based on my review I have a couple of comments anyway below. Please let me know if you want to address these comments (first) or if I can go ahead and start IETF last call?
> 
> Mirja
> 
> ————————————
> AD review comments:
> 
> One high level comment: 
> 
> The intro says:
> "This document only provides broad-level criteria for evaluating a new
>   congestion control algorithm. “
> However, this doc doesn’t really provide that much criteria, it rather specifies evaluation metrics and testing parameter/models. Under criteria I would rather understand something like we have for every test case (“In this scenario, the flows should share the available capacity roughly equally.”). I’m not proposing to extend this doc this way as most criteria might be scenario specific but it could at least say that criteria are further described with the test cases and then the intro could be adapted to better reflect the actual content of the doc.
> 
> 
> 
> Other small technical comments:
> 
> - sec 4.2: Losses of 10% or 20% seems really high. Is that really needed? Where that values even used in any of the results presented?
> 
> - sec 4.4: Do you maybe have a reference for the Gilbert-Elliot model? And what’s exactly meant by "losses generated by modeling a queue including its (different) drop behaviors”? Do you have a reference for that as well? Maybe there are also some IPPM RFCs that could be referred here?
> 
> - I don’t quite understand this sentence in section 6.1:
> “This covers the case where the short TCP flows are not fetching a video file.”
> Wouldn’t it be better to say but case it does covers?
> Also you say file sizes of 30-50 KB: Should they be equally distributed?
> 
> 
> 
> Editorial comments/nits:
> 
> - Sec 3 maybe: s/The following are calculated based on/The following metrics are calculated based on/
> 
> - It doesn’t seem that section 5 is needed at all, given draft-ietf-rmcat-wireless-tests is already mention in the intro.
> 
> - sec 6.2: "The currently chosen video streams are: Foreman and FourPeople.”
> “currently” is a bit weird here, maybe: “The following video streams are recommended for testing.”…?
> 
> - sec 7: s/congestion control protocola/congestion control protocols/
> 
> 
>