Re: [Roll] draft-tripathi-roll-rpl-simulation-06

[Omprakash Gnawali <[e-mail]>]

On Mon, Feb 14, 2011 at 11:30 AM, Jaudelice de Oliveira
<[e-mail]> wrote:
> Hello Omprakash,
> On Feb 13, 2011, at 2:17 PM, Omprakash Gnawali wrote:
>
> On Wed, Feb 2, 2011 at 12:46 AM, JP Vasseur <[e-mail]> wrote:
>
> Dear all,
> Since draft-tripathi-roll-rpl-simulation-06 has been discussed for quite
> some time and is now fully stable, the authors intend to request the
> independent publication of the document. Do not hesitate to chime in if you
> are opposed to it.
>
> I thought there were some serious questions raised about the
> methodology etc. Not sure if they have been addressed.
>
> Can you elaborate a bit on your comment above so I can tell you how the
> document
> was revised accordingly. We did address all comments we were given,
> including yours.
> BTW: thanks again for your feedback.

>From ROLL Virtual Working Group Minutes - June 2010:

"
...
David C.: Can I respond to that? It's wonderful to be driving these decisions
from data. Great to see these results and this process going. To me,
the question isn't whether we should have a simulation working group
document, but in what form. The issue is that the conclusions we reach
are so based on a set of assumptions. Underlying topology, churn, traffic
pattern, etc. What I'd like to see is to build up an understanding of
what needs to be in the WG document to be useful. We should set out what
the simulation methodology is, this is more important than any particular
study.

David C.: That's my biggest concern, going from the individual draft into
a document that the working group could benefit from. There should be
a WG document on anaylsis and how to do this going forward, and I'd want
to hear from the WG an understanding of what is important.
...
"

On this mailing list on 14 Jun 2010 14:04:01 -0700:

"
...
1) My concerns with the methodology remain; assuming independent
packet losses over 10-minute intervals can have significant effects on
how a protocol reacts (e.g., NUD). It makes me unable to make strong
conclusions from any of the rest of the results.

2) "To simulate a more realistic scenario, 20% of the generated
packets by each node are destined to the root, and the remaining 80%
of the packets are uniformly assigned as destined to nodes other than
the root." Can you provide some insight into why this is more
"realistic?" Networks rarely follow uniform distributions, and using
one can make you reach the wrong conclusion (e.g., algorithm X scales
when it doesn't in real patterns).

Note that these are basic concerns with the methodology to obtain the results.

Phil
"
- om_p