[rmcat] Review of draft-ietf-rmcat-eval-test-02

[Safiqul Islam <[e-mail]>]

Hi all,

Here are my comments for eval-test draft. The draft provides sufficient details of the basic test scenarios to evaluate the candidate algorithms.


General comments:

  *   The abstract can clarify the scope of the test scenarios.
  *   The document uses on the following terms interchangeably to represent RMCAT flows. I would like to suggest using only one of them in the document to make it consistent:

     *    Media flow
     *    RMCAT flow
     *    RMCAT media flow

  *   The document mentions “it is expected to log enough information, e.g., Page 14, section 5.3,  “To evaluate the performance of the candidate algorithms it is expected to log enough information “
     *   what is “it” ? It needs to be clarified.
  *   The document suggests to use bit rate range 150 kbps - 1.5 Mbps. I think the upper limit should be increased to match with  real-life applications.


Specific, miscellaneous comments/questions:

  *   Page 3, section 3, Expected behavior:,  describe -> describes
  *   I suggest using either behaviour or behavior to make it consistent.
  *   Page 3, last line, The following basic structure of test case -> of a test case
  *   Page 4. line 2, I suggest using a different verb instead of “flow" to avoid confusion with “flows"
  *   Page 4, line 11, The following sentence needs to be rephrased.

It should be noted that depending on the test cases it is possible to have different path characteristics in of the either directions.

  *   Page 7, section 4.1,  metircs -> metrics
  *   Page 9, section 4.3,  The following sentence needs to be revised.

"Also it is possible that the media traffic generator used in a particular  simulator or testbed if not capable of generating higher bitrate.”

  *   Page 10,  line 2, unless otherwise specified in the test case -> unless otherwise specified in the test case(s)
  *   Page 10, an optimal startup  bitrate values -> an optimal startup  bitrate value
  *   Page 10, section 5.1, maximum Media Bit Rate is Greater than Link Capacity -> maximum media bit rate is greater than link capacity
  *   Page 11, paragraph 2, the following sentence needs to be rephrased to enhance clarity.

        The oscillations occur when the media flow(s) attempts to reach its maximum bit rate, overshoots the usage of the available bottleneck capacity, to rectify it reduces the bit rate and starts to ramp up again.

  *   Page 12, the following sentence can be broken into smaller sentences to enhance clarity.

When using background non-adaptive UDP traffic to induce time-varying bottleneck for the RMCAT flow, the physical path capacity is 4Mbps and the UDP traffic source rate changes over  time as (4-x)Mbps, where x is the bottleneck capacity specified in Table 1

  *   Page 13, at a fine enough time granularity: -> at a fine enough time granularity.
  *   Page 14, i.e, when the backward channel has no impairments -> i.e, when the backward channel has no impairments.
  *   Page 14, section 5.3, at a fine-grained time intervals: -> at a fine-grained time interval.
  *   Page 16, section 5.4, more than one RMCAT media flow shares the -> more than one RMCAT media flow share the

  *   Page 17,  connected to respective R1, R2, R3. -> connected to their respective destinations R1, R2, R3.

  *   Page 19,  It is expected that the competing flows will converge to bit rates to ->  It is expected that the competing flows will converge to optimum bit rates to
  *   Page 19, at a fine enough time granularity: -> at a fine enough time granularity.
  *   Page 19,  One-Way propagation delay for each flow: 10ms, 25ms, 50ms, 100ms, 150ms. -> One-Way propagation delay for each flow: 10ms, 25ms, 50ms, 100ms, and 150ms respectively.
  *   Page 20, section 5.6, at least one long lived TCP flows -> at least one long lived TCP flow
  *   Page 21, connected to corresponding media sink, R1 -> connected to its corresponding media sink, R1
  *   Page 21,  The TCP traffic goes over the forward path from, S_tcp with acknowledgement packets flowing along the backward path from, R_tcp ->  The TCP traffic goes over the forward path from S_tcp, and corresponding acknowledgement packets are transmitted over the backward path from R_tcp.
  *   Page 22, one (1), long-lived TCP -> one (1) and long-lived TCP
  *   Page 22, Congestion control: Default TCP congestion control. -> Congestion control: default TCP congestion control.
  *   Page 22, Test Specific Information: None -> Test Specific Information: none
  *   Page 23, This scenario shows the performance of the multimedia application -> This scenario shows the performance of a multimedia application
  *   Page 23, Media Source:  - > Media source:
  *   Page 24,  The following sentence needs to be revised:
     *   Not all short TCPs start at the same time, 2 start in the ON state while 8 start in an OFF stats.
  *   Page 25, section 5.8, At these stage one of the media flow is -> At this stage, one of the media flows is
  *   Page 25, section 5.8, flows and as they -> flows as they
  *   Page 25. section 5.8,  The following sentence s

When the paused media flow will resume it would no longer have the same bandwidth share on the link.  It has to make  it's way through the other existing flows in the link to achieve a fair share of the link capacity.

  *   Page 25, section 5.8, The general description of the testbed parameters are -> The general description of the testbed parameters is
  *   Page 26, section 6.1, the candidate algorithms does not fail -> the candidate algorithms do not fail
  *   Page 26, section 6.2, In this test case one RMCAT flow, S1->R2 traverse a path -> In this test case one RMCAT flow, S1->R2 traverses a path
  *   Page 27, connected to respective destinations R1, R2, and R3. -> connected to their respective destinations R1, R2, and R3.

  *   I would like to suggest the following two additional test-cases under section 6 (other potential test cases).

6.3 Prioritization test

In this test case, three RMCAT media flows started  out with a priority of 1 each. At around 50 s, the priorities of flow 2 and flow 3 were decreased to 1/3 and 2/3.  A flow i  is assigned the available capacity based on its priority p_{i} as follows:

b_{i} = (p_{i} / Sum(p_{k}) ) * BW   ;  k = {1.. 3}

This test case ensures that flows are getting their assigned portions of the available capacity based on their assigned priorities. It can be useful for [I-D.ietf-rmcat-coupled-cc]. It addresses the requirement 3A in  [I-D.ietf-rmcat-cc-requirements]

Testbed topology: Same as test case defined in section 5.4

Testbed Attribute:

The general description of the testbed parameters is same as Section 5.4 with changes in the test specific setup as  below-

  o  Other test specific setup:

      *  RMCAT media flow timeline:

         +  Flow ID: One (1)

         +  Start time: 0s

         +  Flow duration: 119s

         +  Priority: no change

     *  RMCAT media flow timeline:

         +  Flow ID: Two (2)

         +  Start time: 0s

         +  Flow duration: 119s

         +  Priority:  change to 1/3 after 50s

      *  RMCAT media flow timeline:

         +  Flow ID: Three (3)

         +  Start time: 0s

         +  Flow duration: 119s

         +  Priority: change to 2/3 after  50s


6.4 RMCAT flows competing with one or more long TCP flows

This test case is an extension of the test case defined in section 5.6. It is important for [I-D.ietf-rmcat-coupled-cc], where it requires more than one RMCAT flows in order to couple them.

The general description of the testbed parameters is same as section 5.6.

Regards,
Safiqul