Re: [rmcat] Wireless test cases

["Xiaoqing Zhu (xiaoqzhu)" <[e-mail]>]

​This is my view too.


I still believe that it's important to understand candidate solution behavior over wireless. My colleagues and I are actively working on that.  I just don't think it's realistic to tie the progress of the candidate solution drafts to such evaluations, for two reasons:

a)  the wireless test case draft itself is still evolving;

b)  folks have not yet converged on a common platform (testbed or simulation) for carrying out wireless tests. That makes it hard to compare across candidate solutions.


Thanks,

Xiaoqing

________________________________
From: Stefan Holmer <[e-mail]>
Sent: Monday, February 29, 2016 7:38 AM
To: Mirja Kühlewind; Xiaoqing Zhu (xiaoqzhu)
Cc: Zaheduzzaman Sarker; rmcat WG ([e-mail]); Anna Brunström
Subject: Re: [rmcat] Wireless test cases

I don't think we should require mobile/wireless evaluation results before experimental publication. I think we have proof enough from the base test cases, especially with the extra artificial jitter and fast capacity changes.

/Stefan

On Fri, Feb 26, 2016 at 6:10 PM Mirja Kühlewind <[e-mail]<mailto:[e-mail]>> wrote:
Hi all,

thanks for you feedback.

This brings me back to my original question: Would the working group require to see evaluation results to the mobile and wireless test cases before moving to publication for experimental or are the basic test cases (in draft-ietf-rmcat-eval-test-02) sufficient to move on and classify the candidates to be safe for experimentation in the Internet?

Mirja


> Am 26.02.2016 um 17:50 schrieb Xiaoqing Zhu (xiaoqzhu) <[e-mail]<mailto:[e-mail]>>:
>
> ​I will look into sharing the NS3 code/script for WiFi tests.
>
> As for LTE test cases:  it's good to know that it can be done.  But, since there is no precedence to follow, this may take substantial time and effort. So we may not get around to tackle that soon.
>
> Thanks,
> Xiaoqing
>
> From: Stefan Holmer <[e-mail]<mailto:[e-mail]>>
> Sent: Friday, February 26, 2016 12:55 AM
> To: Zaheduzzaman Sarker; Xiaoqing Zhu (xiaoqzhu); Mirja Kühlewind
> Cc: rmcat WG ([e-mail]<mailto:[e-mail]>); Anna Brunström
> Subject: Re: [rmcat] Wireless test cases
>
> As a first step we'll probably look into at least some of the wifi test cases, but I don't know how soon we can have those done. Our current focus is on improving the loss-based part of GCC, and after that we can focus on further tests.
>
> If it's reasonably simple to run lte tests then it sounds good to run some of those too. If we can share code to set up NS3 for wifi and lte tests, that would be great.
>
> /Stefan
>
> On Fri, Feb 26, 2016 at 8:51 AM Zaheduzzaman Sarker <[e-mail]<mailto:[e-mail]>> wrote:
> Hi,
>
> We didn't used NS-3 simulator to evaluation SCReAM over LTE, we used our proprietary simulator.
> However, I have looked into the NS-3 simulator and talked with the LTE module implementer of NS-3 (LUNA). My conclusion is, the draft can be realized in the NS-3 simulator.
>
> BR
>
> Zahed
>
> > -----Original Message-----
> > From: Xiaoqing Zhu (xiaoqzhu) [mailto:[e-mail]<mailto:[e-mail]>]
> > Sent: den 25 februari 2016 18:43
> > To: Zaheduzzaman Sarker; Mirja Kühlewind
> > Cc: rmcat WG ([e-mail]<mailto:[e-mail]>); Anna Brunström
> > Subject: Re: [rmcat] Wireless test cases
> >
> > Thanks for the pointer, Zahed.  That's a good place to start.  Would it be
> > possible for you to share your NS-3 code/script for testing SCReAM over LTE?
> >
> > BTW, you may want to update the corresponding link in the wireless-test-case
> > document. Currently, the reference on [LTE-simulator] points to the NS-3
> > manual.
> >
> > Best,
> > Xiaoqing
> >
> > ________________________________________
> > From: Zaheduzzaman Sarker <[e-mail]<mailto:[e-mail]>>
> > Sent: Wednesday, February 24, 2016 11:45 PM
> > To: Mirja Kühlewind; Xiaoqing Zhu (xiaoqzhu)
> > Cc: rmcat WG ([e-mail]<mailto:[e-mail]>); Anna Brunström
> > Subject: Re: [rmcat] Wireless test cases
> >
> > Yes,
> >
> > NS-3 has necessary modules to run LTE simulations.
> >
> > https://www.nsnam.org/docs/models/html/lte.html
> >
> > BR
> >
> > Zahed
> >
> > On 02/24/2016 06:02 PM, Mirja Kühlewind wrote:
> > > Hi Xiaoqing,
> > >
> > >> Am 24.02.2016 um 17:50 schrieb Xiaoqing Zhu (xiaoqzhu)
> > <[e-mail]<mailto:[e-mail]>>:
> > >>
> > >> Hi Mirja, all,
> > >>
> > >> Zahed and I have indeed discussed about this in Yokohama. Maybe I did not
> > make it clear in my previous email, trying again.
> > >>
> > >> Basically, to carry out tests in Sec 3 (Cellular Network Specific Test Cases),
> > one needs to run it over a module that simulates packet behavior over cellular
> > links -- just like for the Wi-Fi portion of the test cases, we relied on the Wi-Fi
> > module in NS3 simulators to do so.
> > >>
> > >> What is missing is the cellular network simulator, preferably as a plug-in
> > module for NS3, over which we can run rate adaptation tests.
> > > I though that this is available in the mean time.
> > >
> > > Zahed, do you have link or is this already included in the latest version of ns-
> > 3?
> > >
> > > Mirja
> > >
> > >
> > >> Thanks,
> > >> Xiaoqing
> > >>
> > >>
> > >> ________________________________________
> > >> From: Mirja Kühlewind <[e-mail]<mailto:[e-mail]>>
> > >> Sent: Wednesday, February 24, 2016 3:33 AM
> > >> To: Xiaoqing Zhu (xiaoqzhu); rmcat WG ([e-mail]<mailto:[e-mail]>)
> > >> Cc: Anna Brunström; Zaheduzzaman Sarker
> > >> Subject: Re: [rmcat] Wireless test cases
> > >>
> > >> Hi Xiaoqing,
> > >>
> > >> On 19.02.2016 21:27, Xiaoqing Zhu (xiaoqzhu) wrote:
> > >>> As for Sec. 3 on Cellular Network Specific Test Cases, would
> > >>> appreciate more detailed information on how to set those up, or
> > >>> relevant pointers/code on cellular network simulators.
> > >> Can you let us/Zahed know what exactly you need/is missing?
> > >>
> > >> Thanks,
> > >> Mirja
> >
>