Re: [Roll] Fwd: New Version Notification for draft-tripathi-roll-rpl-simulation-03

[Philip Levis <[e-mail]>]

On Mar 31, 2010, at 8:31 PM, Joydeep Tripathi wrote:

> On Tue, Mar 30, 2010 at 1:29 PM, Philip Levis <[e-mail]> wrote:
>> On Mar 30, 2010, at 12:46 AM, Joydeep Tripathi wrote:
>> 
>>> Hi,
>>> 
>>> We published another draft on Simulation results of RPL. In this
>>> revision, we added path stretch and delay metric comparison of the
>>> packets in RPL routing with shortest path routing. We also added
>>> Simulation results of RPL in a typical home/building routing traffic
>>> scenario. Thanks to Jerry for providing the details of message length
>>> and distribution in such a network. Note, we did not attempt to use
>>> any random topology with a random link quality, cause that might not
>>> provide us with intuition into how real deployment may behave in
>>> presence of link quality variation and link PDR distribution. We also
>>> simulate RPL with a 86 node topology, and in process of simulating it
>>> in another real network of few thousand nodes.
>>> 
>>> This simulation study shows, the path stretch is very less in
>>> the home/building routing scenario. Also, the delay bound in RPL is
>>> not worse than that of shortest path routing, showing P2P routing in
>>> RPL a viable option for this kind of applications in terms of Path quality.
>>> 
>>> Looking forward to any comments/suggestions.
>>> 
>> 
>> Joydeep,
>> 
>> I have concerns with the simulation methodology. In particular:
>> 
>> 1) Links are selected randomly. Links in real networks can be correlated in behavior, either due to hardware variations or correlated environmental effects (e.g., interference). The problem with random selection is that as a node has more links, it will inevitably have some good ones. In practice this isn't always true (e.g., a node is near an interference source). This will lead the simulation results to think that RPL behaves better than it does in practice.
> 
> Hi,
> 
> I understand your concern. But these link data have been gathered from
> a real network. We will also try acquiring more data, where the link
> PDR may vary much more faster. We are also in the process a bigger
> network data and come up with simulation result for a bigger scale
> real network with real PDR variation.

Joydeep,

I realize those data have been collected from a real network. I collect data from real networks too. :)

The issue is not a network where the link PDR varies much faster: the issue is the interval over which you compute the PDR. Do you have experimental evidence that links exhibit iid losses for 10 minute stretches? This goes against most measurement studies and RF communication theory.

The question here isn't size or scale: it's accuracy. Smoothing over details such as these can lead to incorrect conclusions. E.g., if a link has independent losses, the retransmissions can be used for arbitrary reliability. A simulation could demonstrate perfect reliability with low latency, something which clearly isn't true in practice.

Phil