Re: [rmcat] Ongoing working group last calls

[Sergio Mena <[e-mail]>]

RMCAT working group,

I could finally carve out some time to review 
*draft-ietf-rmcat-eval-test-06*.

The last time read this draft thoroughly was version -01. So what 
follows is based on reviewing the text differences between version -01 
and version -06.

I have classified my feedback into minor feedback (mostly editorial), 
and "less minor".

  * Minor/editorial:
      o Page 3. Last paragraph. "These sources generate media traffic
        and use either congestion control algorithm under
        investigation". Consider changing "either" to "one of the", or
        "any"; as there are (or might be) more than two algorithms.
      o Page 4. Beginning of paragraph just below Figure 1. "In a
        testbed environment where real equipments are used to create a
        laboratory,". This statement's wording sounds strange to me...
        did you mean to say something like the sentence below ?
        "In a testbed environment where real equipment is set up in a
        laboratory network, "
      o Page 4. Same paragraph as previous bullet. Consider changing
        "test bed" to "testbed"
      o Page 8. "A.  End-to-end delay for the congestion controlled
        media flow.". Consider changing "flow" by "flow(s)" for
        consistency with E below
      o Page 17. First paragraph. Consider changing "more than one media
        flows" to "two or more media flows"... alternatively, consider
        changing "flows" to its singular form.
      o Page 19. 4th paragraph. I think mentioning the propagation
        delays here is redundant, as they are specified a few lines below.
      o Page 25. Consider changing "start in an OFF stats" by "start in
        an OFF state" (or maybe I misread something)
      o Page 27. Second last paragraph. I don't really understand what
        "flow S1->R2" means... did you mean S1->R1 ?
  * Less minor:
      o Page 12. "One-way propagation delay: [50 ms, 100 ms]. on the
        forward path direction". One should assume then that the one-way
        propagation delays will be different for forward and backwards
        paths, as the backward path's propagation delay is not mentioned
        (so doesn't change)... is this intended? A clarification would
        be good
      o Page 12. Last paragraph. I see Table 1 has been updated to
        contain capacity factors, rather than absolute values. I'm fine
        with that. Bearing that in mind, I agree with the formula
        (4-x)Mbps; however, the explanation of x still looks like it
        denotes an absolute value (not a factor). I would propose
        something like "(4-x)Mbps, where x is the path capacity ratio
        specified in Table 1"
      o Page 14. Second paragraph. "The reference bottleneck capacity is
        2Mbps [...] source rate changes over time as (4-x)Mbps". I think
        that is inconsistent, as it should read (4-2*x)Mbps, given that
        the reference bottleneck capacity is 2Mbps in this test case.
        Also, same comment about the explanation of x applies here (it
        gives the impression x is an absolute value).
      o Page 16. Same comment on explanation of 'x'
      o Page 19. First paragraph. "end-to-end path latency": I don't
        seem to find this term defined anywhere in the draft. If think I
        should be clearly defined, to avoid different interpretations of
        that it is (e.g., as I interpret it: the delay of a packet
        traveling end to end in the absence of any other traffic).
        Moreover, a few lines below the term "One-Way propagation delay"
        is used. I would unify those two terms (if I didn't
        misunderstand the text, they denote the same concept).
      o Page 22. Section 5.6. Consider making the test duration higher
        (e.g. 300 seconds, just as test case 5.7). In the last months
        (or years), we have tested NADA extensively against TCP using
        this test case, and we found 120 seconds to be too short to
        fully appreciate the behavior of the algorithms (TCP and
        candidate) competing for the bandwidth. We ended up increasing
        the test duration every time we wanted to analyze the results of
        this test case.
      o Page 22. Section 5.6. Consider removing bottleneck queue size of
        1000ms from the test, as 1-second queuing delay while competing
        with TCP (i.e., loss-based) renders the real time media flow
        unusable (unmanageable delay for the user), even if the
        algorithm manages to get its fair share of the bottleneck bandwidth.
      o Page 25. Section 5.7. Consider modifying the test case so that
        the short lived flows stop some time before the end of the test
        (e.g., 60 seconds before the end). This modification permits the
        test to show whether the congestion controlled flows are able to
        come back to normal behavior after all short-lived TCP flows
        have stopped (note that the short-lived TCP flows overshoot the
        bottleneck quite a lot with their current parameters). Quite
        frankly, this is one of the points that makes this test case a
        meaningful one. I remember the rmcat group members discussing
        about the usefulness of this test case in the past. With the
        modification I'm proposing here, it's definitely useful to me.

That's all I have. I hope this feedback helps improve the quality of the 
draft.

Thanks,

Sergio


On 06/08/18 17:52, Colin Perkins wrote:
> RMCAT working group,
>
> As noted at the meeting in Montreal, the following drafts are in working group last call:
>
> - draft-ietf-rmcat-eval-criteria-07
> - draft-ietf-rmcat-eval-test-06
> - draft-ietf-rmcat-video-traffic-model-04
>
> All three working group last calls were due to conclude at that meeting, but we have not yet received sufficient reviews of these drafts to do so.
>
> If we are to progress these drafts, we need additional reviews from working group members. Please, do read the drafts. If you have no comments and think the drafts are ready to progress, then please send a note to that effect. If you have comments, please do send them to the list.
>
> Thanks,
> Colin (as WG co-chair)
>
>
>
>
>