Re: [rmcat] test cases for coupled cc [was: Re: Review of draft-ietf-rmcat-eval-test-02]

[Zaheduzzaman Sarker <[e-mail]>]

Hi,

Please see inline.

BR

Zahed

On 02/10/2016 03:36 PM, Safiqul Islam wrote:
> HI Zahed, Hi Mirja,
>
> Please see inline
>
> /Safiqul
>
>
>> On 10. feb. 2016, at 10.24, Zaheduzzaman Sarker 
>> <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Hi Mirja,
>>
>> please see inline....
>>
>> On 2016-02-09 15:51, Mirja Kühlewind wrote:
>>>>
>>>> The test cases in the draft were designed for end-point adaptation
>>>> algorithm without coupled congestion control in mind i.e. no need
>>>> for shared bottleneck detection and flow priority. In the last IETF
>>>> meeting we talked about the possibility of extending the test cases
>>>> to include coupled congestion control algorithm.
>>>
>>> I believe we decided already at the last meeting that we need to
>>> adapt this draft to include uses cases for coupled congestion (and
>>> that’s why we asked Safiqul to propose some).
>>
>> That is what I can also recall but didn't see anything in the meeting
>> minutes so was not sure what we agreed on. May be missed it in the 
>> meeting minutes.
>>
>>>
>>>>
>>>> Now if we extend the test cases then I don't think only adding the
>>>> two proposed test cases will be sufficient.
>>>
>>> Yes, that might be true but adding these use cases below would
>>> probably be a good starting point.
>>
>> If I understand shafiqul's last mail, it is fine with only having 
>> test results with priority for 5.4 test case.
>
> For simplicity, I suggested to use only test case 5.4, but I am not 
> against using other cases.
> I can add some recommendations for other cases too.
A section in 5.4 describing the required changed to run with couple CC 
should be enough.
>
>>
>>>
>>>> We will need to identify all the test cases those are suitable for
>>>> running with coupled congestion control.
>>>
>>> Not sure, if you only have one flow that test case is simply not
>>> suitable and that’s fine.
>> Yes, that is one identification criteria. But may be we dont need 
>> results for all the test cases who has more than one flow. see my 
>> previous comment.
>
> +1
>
>
>>>
>>>> Like 5.2 can be run with priority per flow. By doing this we can
>>>> mention (in each test case or in one place in the document) that
>>>> one can run tests with flow priority and perhaps give some means to
>>>> evaluate the available capacity distribution with some equation
>>>> like the one mentioned here in the proposed test case 6.3.
>>>
>>> Yes.
>>>
>>>> However, then this document also needs to include description for
>>>> shared bottleneck detection.
>>>
>>> Why?
>> Bcos I dont think the document now even mention "shared bottleneck" 
>> and to me coupled congestion control does not make sense without 
>> shared bottleneck.
>
> The document covers the basic scenarios where flows share a common 
> bottleneck.  We can simply add a line saying “these additional 
> coupled-cc test-cases are used for flows sharing a bottleneck”.
yes, something like that.
>
>>>
>>>> In the simplest form, we can perhaps assume that the identified
>>>> test cases have single bottleneck point.
>>>
>>> Yes.
>>>
>>>>
>>>> I think we need to discuss what is best possible way to extend the
>>>> test cases if we agreed to do that.
>>>
>>> As I said I think we agreed already. Please following with Safiqul on
>>> this!
>>>
>>>>
>>>> But before that I would like to know if the WG is OK with extending
>>>> the test cases for coupled congestion control.
>>>
>>> Any further feedback on this from the group is of course welcome!
>>> Please speak up now!
>>>
>>>>
>>>> And will it be a compulsory (candidate algorithms MUST present
>>>> results) or optional (candidate algorithms MAY present results)
>>>> part of the test case.
>>>
>>> I guess only candidates that use coupled cc..?
>>
>> hmm...
>>
>> I am a bit confused now. Are we talking about test case(s) to 
>> evaluate the Coupled CC or the use of Coupled CC in the candidate CC?
>
>
> Without making it compulsory, I suggested to add these in the other 
> cases. We can perhaps say:
>
> “When flows share a common bottleneck, combining their congestion 
> controllers can be beneficial.
> These additional test cases can be used to evaluate a congestion 
> control mechanism when using a method to couple flows, such as 
> [I-D.ietf-rmcat-coupled-cc]."
>
> Does this put it better?
I dont think this document to referent to any candidate solution.

I think the test cases should be defined so that one can evaluate other 
alternatives if there is any. Now, if there is more than one coupled CC 
solution and the candidate algorithms perform differently with different 
solution, how that will help us to evaluate coupled Congestion control 
algorithms. How does the WG plans to solve such situation?