[Tmrg-interest] Tools for the Evaluation of Simulation and Testbed Scenarios

[[e-mail] (Sally Floyd)]

We have submitted a *very* rough draft of a new TMRG internet-draft,
Tools for the Evaluation of Simulation and Testbed Scenarios, and
there is a local copy at the URL below:

 http://www.icir.org/tmrg/draft-irtf-tmrg-tools-00.txt
 http://www.icir.org/tmrg/draft-irtf-tmrg-tools-00.ps

The abstract is as follows:

    This document describes tools for the evaluation of simulation and
    testbed scenarios used in research on Internet congestion control
    mechanisms.  We believe that research in congestion control
    mechanisms has been seriously hampered by the lack of good models
    underpinning analysis, simulation, and testbed experiments, and that
    tools for the evaluation of simulation and testbed scenarios can
    help in the construction of better scenarios, based on better
    underlying models.  One use of the tools described in this document
    is in comparing key characteristics of test scenarios with known
    characteristics from the diverse and ever-changing real world.
    Tools characterizing the aggregate traffic on a link include the
    distribution of per-packet round-trip times, the distribution of
    packet sequence numbers, and the like.  Tools characterizing end-to-
    end paths include drop rates as a function of packet size and of
    burst size, the synchronization ratio between two end-to-end TCP
    flows, and the like.  For each characteristic, we describe what
    aspects of the scenario determine this characteristic, how the
    characteristic can affect the results of simulations and experiments
    for the evaluation of congestion control mechanisms, and what is
    known about this characteristic in the real world.  We also explain
    why the use of such tools can add considerable power to our
    understanding and evaluation of simulation and testbed scenarios.

I am sending the draft to the mailing list at this very early state
to solicit volunteers to provide text, and/or citations to the most
recent literature, for the various sections in the document.  One
goal is to clarify what measurements are already largely in place
(e.g., the distribution of connection sizes, or of round-trip times)
and what measurements are desperately needed (e.g., the packet drop
rate as a function of packet size, or the synchronization ratio for
high-bandwidth protocols over realistic high-bandwidth networks.)

Any general feedback about the overall direction of the draft would
also be useful.

My hope is for the group to use the information in this draft, as
it progresses, to guide the collection and creation of
best-current-practice simulation and testbed scenarios for evaulating
congestion control mechanisms.

Many thanks.

- Sally
http://www.icir.org/tmrg/