Re: [rmcat] AD review of draft-ietf-rmcat-eval-criteria-10

Joerg Ott <ott@in.tum.de> Fri, 14 February 2020 17:25 UTC

Return-Path: <ott@in.tum.de>
X-Original-To: rmcat@ietfa.amsl.com
Delivered-To: rmcat@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E5A5D120923; Fri, 14 Feb 2020 09:25:44 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.219
X-Spam-Level:
X-Spam-Status: No, score=-4.219 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SR5102MrZIMR; Fri, 14 Feb 2020 09:25:39 -0800 (PST)
Received: from mail-out2.informatik.tu-muenchen.de (mail-out2.in.tum.de [131.159.0.36]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8EF7C120935; Fri, 14 Feb 2020 09:25:36 -0800 (PST)
Received: by mail.in.tum.de (Postfix, from userid 107) id 7A6901C07E7; Fri, 14 Feb 2020 18:25:34 +0100 (CET)
Received: (Authenticated sender: ott) by mail.in.tum.de (Postfix) with ESMTPSA id 06FE51C07DF; Fri, 14 Feb 2020 18:25:31 +0100 (CET) (Extended-Queue-bit tech_ybujy@fff.in.tum.de)
To: Mirja Kuehlewind <ietf@kuehlewind.net>
Cc: draft-ietf-rmcat-eval-criteria.all@ietf.org, rmcat@ietf.org
References: <3B17A1D8-6743-4513-853B-F8FEAFE604DC@kuehlewind.net> <26540002-af68-478b-face-a415e91a9252@in.tum.de> <3750993F-0A89-43C5-A327-AA6F805C3170@kuehlewind.net> <690cdf0d-e682-d4ba-f057-0f6e3bc07429@in.tum.de> <ECD50DDE-E1F9-4CCD-A73E-C547581015D2@kuehlewind.net>
From: Joerg Ott <ott@in.tum.de>
Message-ID: <e4365828-7342-b80c-688f-e66b42e03350@in.tum.de>
Date: Fri, 14 Feb 2020 18:25:31 +0100
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.4.2
MIME-Version: 1.0
In-Reply-To: <ECD50DDE-E1F9-4CCD-A73E-C547581015D2@kuehlewind.net>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/rmcat/9nEUTa6hbQZDjPgnFYUCm0VZKEU>
Subject: Re: [rmcat] AD review of draft-ietf-rmcat-eval-criteria-10
X-BeenThere: rmcat@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "RTP Media Congestion Avoidance Techniques \(RMCAT\) Working Group discussion list." <rmcat.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/rmcat>, <mailto:rmcat-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/rmcat/>
List-Post: <mailto:rmcat@ietf.org>
List-Help: <mailto:rmcat-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/rmcat>, <mailto:rmcat-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Feb 2020 17:25:45 -0000

Ok, I will figure something out, most likely driven by time budget...

On 13.02.20 18:24, Mirja Kuehlewind wrote:
> I’d be okay if you post an immediate update (if the intention is to remove some or most of appendix A) as having the appendix might be confusing as well. If there is no intention to remove the appendix (but maybe just rephrase it slightly) it could be more appropriate to do it afterwards… it's on you!
> 
> 
> 
>> On 13. Feb 2020, at 18:21, Joerg Ott <ott@in.tum.de> wrote:
>>
>> Hi Mirja,
>>
>> oops, my bad, I briefly looked at the other email, too, but thought
>> there wasn't much additional stuff in there.  Sorry.  I can do the
>> update as part of the LC process then.  To avoid confusion, I won't
>> post an immediate update (for some definition of immediate).
>>
>> Best,
>> Jörg
>>
>> On 12.02.20 23:24, Mirja Kuehlewind wrote:
>>> Hi Joerg,
>>> Thanks for these updates. However, there were actually additional comments I’ve sent in a second mail right afterwards. Here are the comments again for your convince (mostly about references):
>>> -----
>>> Actually one more question: There is appendix A which reads like there is still work to do. Maybe that is information that should be rephrased and added in the intro?
>>> Further the terminology section (2) references to a bunch of RTP document normatively. However, I don’t think this terminology is actually used in this draft. Can you please double-check and adapt the references accordingly!?
>>> And again about the spilt between normative and informational. I think this document does not need to have the test RFCs as normative references (but only the other way around: test RFCs reference this doc normatively). However, given section 6.2., RFC8593 should probably be a normative reference.
>>> And as I’m looking at the references: I think section 3 could probably reference more closely some sections of RFC5166.
>>> ——
>>> I will request the IETF last call now nevertheless. You can fix these things after the last call or any time during the last.
>>> Thanks!
>>> Mirja
>>>> On 12. Feb 2020, at 17:03, Joerg Ott <ott@in.tum.de> wrote:
>>>>
>>>> Hi Mirja,
>>>>
>>>> so, finally, we are getting there now that teaching is over...
>>>>
>>>>> One high level comment:
>>>>> The intro says:
>>>>> "This document only provides broad-level criteria for evaluating a new
>>>>>     congestion control algorithm. “
>>>>> However, this doc doesn’t really provide that much criteria, it rather specifies evaluation metrics and testing parameter/models. Under criteria I would rather understand something like we have for every test case (“In this scenario, the flows should share the available capacity roughly equally.”). I’m not proposing to extend this doc this way as most criteria might be scenario specific but it could at least say that criteria are further described with the test cases and then the intro could be adapted to better reflect the actual content of the doc.
>>>>
>>>> The "criteria" (after which the doc was named originally) has probably
>>>> historic roots and specific criteria (and threshold, etc.) got lost
>>>> during splits over time.  I fixed the wording to no longer make an
>>>> explicit reference to criteria but just point to the specific other
>>>> documents for test cases.
>>>>
>>>>> Other small technical comments:
>>>>> - sec 4.2: Losses of 10% or 20% seems really high. Is that really needed? Where that values even used in any of the results presented?
>>>>
>>>> We expect all IETF protocols to gracefully degrade in response to
>>>> increasingly severe situations.  This also implies avoiding that
>>>> protocols exhibit strange behavior once they leave their primary
>>>> operational range.  By extending the parameter space beyond just
>>>> sunshine conditions, we ensure that also less favorable conditions
>>>> are validated and compared.
>>>>
>>>> I could add a note to this end, but I do not want to create the
>>>> assumption of some values being of secondary relevance only.
>>>> So will leave this as is.
>>>>
>>>>> - sec 4.4: Do you maybe have a reference for the Gilbert-Elliot model? And what’s exactly meant by "losses generated by modeling a queue including its (different) drop behaviors”? Do you have a reference for that as well? Maybe there are also some IPPM RFCs that could be referred here?
>>>>
>>>> I could include a reference here for Gilbert-Elliot such as:
>>>> http://www.ohohlfeld.com/paper/hasslinger_hohlfeld-mmb_2008.pdf
>>>>
>>>> However, this not mandatory text and the second part of the sentence
>>>> is even less specific and does (deliberately) not give a specific
>>>> example.  We don't want to create any prejudice here beyond random
>>>> losses.  The Internet changes, after all.
>>>>
>>>> As for IMPP, do you have specific ones in mind?  I am not familiar
>>>> with all IMPP work except that I looked a metrics -- but not loss
>>>> or other models.
>>>>
>>>> Whatever we extend here may need more serious consideration and
>>>> weighting.  And going back to the WG.  Given that none of this would
>>>> be mandatory, we may opt not specify this in detail.
>>>>
>>>> IIRC, the above text came from comments from the group beyond random
>>>> losses but without a clear mandate of what to pick.
>>>>
>>>>> - I don’t quite understand this sentence in section 6.1:
>>>>> “This covers the case where the short TCP flows are not fetching a video file.”
>>>>> Wouldn’t it be better to say but case it does covers?
>>>>> Also you say file sizes of 30-50 KB: Should they be equally distributed?
>>>>
>>>> Fixed.
>>>>
>>>> I also looked again at the reference provided in the text.  This does no
>>>> longer yield the necessary resolution detail below 100K.  but the mean
>>>> still holds, yielding about 30 KB per request.
>>>>
>>>>> Editorial comments/nits:
>>>>> - Sec 3 maybe: s/The following are calculated based on/The following metrics are calculated based on/
>>>>
>>>> Done.
>>>>
>>>>> - It doesn’t seem that section 5 is needed at all, given draft-ietf-rmcat-wireless-tests is already mention in the intro.
>>>>
>>>> Agree and removed.  Also, the wireless draft is already referred to in
>>>> the introduction, which I kept.
>>>>
>>>>> - sec 6.2: "The currently chosen video streams are: Foreman and FourPeople.”
>>>>> “currently” is a bit weird here, maybe: “The following video streams are recommended for testing.”…?
>>>>
>>>> Yupp, fixed.
>>>>
>>>>> - sec 7: s/congestion control protocola/congestion control protocols/
>>>>
>>>> Fixed -- and also fixed another typo around there.
>>>>
>>>> Thanks much, Mirja, will submit this update now.
>>>>
>>>> Best,
>>>> Jörg
>>>>
>>>>
>>
>