Re: [Ospf-manet] MPR-OSPF GTNetS simulations report

Richard Ogier <rich.ogier@earthlink.net> Mon, 05 February 2007 16:08 UTC

Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1HE6OX-0003vt-4F; Mon, 05 Feb 2007 11:08:41 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1HE6OW-0003vo-Eb for ospf-manet@ietf.org; Mon, 05 Feb 2007 11:08:40 -0500
Received: from pop-knobcone.atl.sa.earthlink.net ([207.69.195.64]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1HE6OV-00010u-6k for ospf-manet@ietf.org; Mon, 05 Feb 2007 11:08:40 -0500
Received: from dialup-4.246.93.156.dial1.sanjose1.level3.net ([4.246.93.156]) by pop-knobcone.atl.sa.earthlink.net with esmtp (Exim 3.36 #1) id 1HE6OP-0003ti-00; Mon, 05 Feb 2007 11:08:35 -0500
Message-ID: <45C75684.5060107@earthlink.net>
Date: Mon, 05 Feb 2007 08:08:36 -0800
From: Richard Ogier <rich.ogier@earthlink.net>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Aniket Desai <adesai@opnet.com>
Subject: Re: [Ospf-manet] MPR-OSPF GTNetS simulations report
References: <E1HBJri-0004Gw-Et@megatron.ietf.org> <6.2.3.4.2.20070131171954.0284c1a8@mailserver.opnet.com>
In-Reply-To: <6.2.3.4.2.20070131171954.0284c1a8@mailserver.opnet.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 538aad3a3c4f01d8b6a6477ca4248793
Cc: ospf-manet@ietf.org
X-BeenThere: ospf-manet@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions of OSPFv3 extensions supporting MANET <ospf-manet.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/ospf-manet>, <mailto:ospf-manet-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/ospf-manet>
List-Post: <mailto:ospf-manet@ietf.org>
List-Help: <mailto:ospf-manet-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/ospf-manet>, <mailto:ospf-manet-request@ietf.org?subject=subscribe>
Errors-To: ospf-manet-bounces@ietf.org

> Hello,
>
> From the previous emails, it seemed that Dr. Ogier suggested the use 
> of simulations as a "tie breaker" to even administer a proposal to 
> experimental status. Simulations may not be the most accurate 
> representation of the reality to the last bit, but they provide 
> sufficient framework to cheaply evaluate and compare different 
> proposals against each other. I believe it should be possible to 
> design a series of parametric runs (changing node density, changing 
> mobility, changing network size, changing radio range etc etc.) to 
> extensively compare all the proposals against each other. All the 
> criticism about considering only a single/limited set of scenarios 
> gets nullified when one has conducted a parametric run, and I don't 
> see any way other than the simulations to do this.
>
> Thanks,
>
> Aniket 


Aniket,

I agree that simulations should be a part of the evaluation process.
Most people agreed at the last IETF meeting that simulations
are important, so we already have consensus on this point.

For example, if implementations show that all solutions perform
about the same for 50 nodes in a number of scenarios, but simulations
show that one scales much better to 100 or more nodes, then that is
important evidence that should be considered, and could be used
to "break a tie".

Simulations are not perfect, but will have some inaccuracy.
For example, if a simulation model is found to predict
overhead with 25% accuracy (compared to a real implementation),
and simulation results show that protocol A has 75% less overhead
than protocol B, that is a significant result.

It is interesting that Emmanuel is presenting results for at
most 50 nodes, whereas it has already been shown that MDR is
scalable to more than 100 nodes.  50-node networks are not
very challenging, now that we can support 100 nodes.
We need to simulate larger networks in order to stress the
protocols and determine which protocols are really scalable.

I ran some simulations with up to 100 nodes, using Emmanuel's
latest code release.  I will present these results in my next
message.

Richard




_______________________________________________
Ospf-manet mailing list
Ospf-manet@ietf.org
https://www1.ietf.org/mailman/listinfo/ospf-manet