Re: [rmcat] Sections 5.1 and 5.2 of draft-ietf-rmcat-eval-test-01

["Xiaoqing Zhu (xiaoqzhu)" <[e-mail]>]

I concur with Sergio’s suggestions.

One thought: instead of stating variation within each sub-section, we can state that there exists two options of changing bandwidth in the beginning of Sec 5. This will also be consistent with the option/flexibility we now offer for folks to pick different initial capacities and scale rest of the BW values accordingly.

Best,
Xiaoqing


From: "Sergio Mena de la Cruz (semena)" <[e-mail]<mailto:[e-mail]>>
Date: Wednesday, August 26, 2015 at 12:35 AM
To: Zaheduzzaman Sarker <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: Re: Sections 5.1 and 5.2 of draft-ietf-rmcat-eval-test-01


Hi Zahed,

First of all, your question. By "insensitive traffic" I meant that the flow is CBR, so it won't back off whether other traffic is present in the bottleneck link or not. Maybe we can find a better name for this...

On the other hand, in the last days, I have been working on an ns3 implementation of some of the test cases in the draft. I realized that it would be coherent (and useful, IMO) for 5.3 to also have a "b" version where both forward and backward paths are changed by a CBR, similarly to 5.1 and 5.2. Let me know your thoughts on this.

Regards,

Sergio



On 24/08/15 13:45, Zaheduzzaman Sarker wrote:
Hi Sergio,

Thanks for your effort.

I think the proposed changes looks good.

As we have plan to update the variation of path capacity relative to initial path capacity for test case 5.1 and test case 5.2, we need to change the respective text in the 5.1b and 5.2b to be aligned with that.

One clarification question : in 5.1b what does the “insensitive traffic” means in the new text?

BR

Zahed

From: Sergio Mena [mailto:[e-mail]]
Sent: den 13 augusti 2015 14:30
To: Zaheduzzaman Sarker
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Sections 5.1 and 5.2 of draft-ietf-rmcat-eval-test-01

Zahed,

[CCed the author list of the rmcat test cases draft, as well as the rmcat mailing list]

There was a discussion triggered at rmcat main session in Prague on how to simulate bottleneck-link bandwidth changes. Two options were discussed (1) changing the physical link throughput (currently specified in the draft), and (2) using an insensitive CBR UDP flow as background traffic.

My understanding from the discussion during the session was that none of these two options can simulate the other and, therefore, we somehow need to test how candidate algorithms adapt in both cases.

You and I then had an offline discussion on the subject in which you suggested me to review sections 5.1 and 5.2 of the test cases draft, and propose an alternate text to extend these two test cases to the background traffic option.

I have reviewed sections 5.1 and 5.2 and come up with a proposal for extending them to using insensitive background traffic, in addition to the current option in the text (physical link throughput change).

Please find below what I would like to propose as updated text for 5.1 and 5.2. [Text in square brackets are instructions to locate and replace the original text].

Thanks,

Sergio Mena


PS Proposed changes:

===========
Section 5.1
===========

[In the three-bullet list after the first paragraph, change the text in the second bullet from:]

    o change in available capacity (e.g., due to interface change,
      routing change).

[to]

    o change in available capacity (e.g., due to interface change,
      routing change, sharp change in background traffic).

---

[Replace the paragraph immediately after the three-bullet list mentioned above, the one that starts with]

It should be noted that the exact variation...

[with the following]

The test case defined in this section (5.1) is divided into two subcases 5.1a and 5.1b. The main
difference is how the bottleneck-link capacity is changed in the testbed. In 5.1a, the goal is to
simulate a modification in the underlying physical properties, such as routing change or interface
change. In 5.1b, the goal is to simulate a sharp change in background traffic to which the candidate
algorithm is to adapt.

---

[Change the last item of testbed attributes from]

      *  Competing traffic:

         +  Number of sources : Zero (0)

[to]

      *  Competing traffic:

         +  In 5.1a

            - Number of sources : Zero (0)

         +  In 5.1b

            - Number of sources : One (1)

            - Type of sources : CBR traffic

            - Sending bitrate : See test specific information below.

            - Traffic direction : Forward

            - Congestion control : None (insensitive traffic)

            - Traffic timeline:

              - Start time : 0s

              - End time : 100s

---

[Change 2nd bullet of test-specific information from]

      *  This test uses bottleneck path capacity variation as listed in
         Table 1

[to]

      *  This test uses bottleneck path capacity variation as listed in
         Table 1.

         - In 5.1a, the path capacity transitions are achieved by instantly changing the
           physical throughput of the bottleneck link. For simplicity, when changing
           the bottleneck link throughput, other parameters such as queue length in
           packets will be left unchanged, even if it implies that the new queue has a
           different length in milliseconds.

         - In 5.1b, the path capacity transitions are achieved by instantly changing the
           sending bitrate of the competing CBR flow. The new sending bitrate must be
           the difference between the bottleneck link-capacity (4 Mbps, as specified in
           4.2) and the capacity specified in Table 1.

           The sharp bitrate change in the CBR flow might imply the losses on both the
           application-related traffic and the CBR flow. As a result, the reaction of
           the candidate algorithm might be different in 5.1a and in 5.1b, hence the
           distinction between the two sub-cases.


===========
Section 5.2
===========

[Insert this after the first paragraph of the section]

Analogously to Section 5.1, the present test case (5.2) is divided into two subcases 5.2a and 5.2b.
The main difference is the same: 5.2a simulates a change in the underlying physical network properties,
such as routing change or interface change; whereas 5.2b simulates a sharp change in background
competing traffic.

---

[Change last paragraph from]

   Test Specific Information: This test uses path capacity variation as
   listed in Table 2 with a corresponding end time of 125 seconds.

[to]

   Test Specific Information: This test uses path capacity variation as
   listed in Table 2 with a corresponding end time of 125 seconds. The
   two media flows stop at time 124.