Re: [Ospf-manet] MPR-OSPF GTNetS simulations report

> Hello,
>
> From the previous emails, it seemed that Dr. Ogier suggested the use 
> of simulations as a "tie breaker" to even administer a proposal to 
> experimental status. Simulations may not be the most accurate 
> representation of the reality to the last bit, but they provide 
> sufficient framework to cheaply evaluate and compare different 
> proposals against each other. I believe it should be possible to 
> design a series of parametric runs (changing node density, changing 
> mobility, changing network size, changing radio range etc etc.) to 
> extensively compare all the proposals against each other. All the 
> criticism about considering only a single/limited set of scenarios 
> gets nullified when one has conducted a parametric run, and I don't 
> see any way other than the simulations to do this.
>
> Thanks,
>
> Aniket 

Aniket,

I agree that simulations should be a part of the evaluation process.
Most people agreed at the last IETF meeting that simulations
are important, so we already have consensus on this point.

For example, if implementations show that all solutions perform
about the same for 50 nodes in a number of scenarios, but simulations
show that one scales much better to 100 or more nodes, then that is
important evidence that should be considered, and could be used
to "break a tie".

Simulations are not perfect, but will have some inaccuracy.
For example, if a simulation model is found to predict
overhead with 25% accuracy (compared to a real implementation),
and simulation results show that protocol A has 75% less overhead
than protocol B, that is a significant result.

It is interesting that Emmanuel is presenting results for at
most 50 nodes, whereas it has already been shown that MDR is
scalable to more than 100 nodes.  50-node networks are not
very challenging, now that we can support 100 nodes.
We need to simulate larger networks in order to stress the
protocols and determine which protocols are really scalable.

I ran some simulations with up to 100 nodes, using Emmanuel's
latest code release.  I will present these results in my next
message.

Richard

_______________________________________________
Ospf-manet mailing list
Ospf-manet@ietf.org
https://www1.ietf.org/mailman/listinfo/ospf-manet