Re: [bmwg] WG Last Call: OSPF convergence benchmarking

[Scott Poretsky <[e-mail]>]

At 07:47 AM 5/16/2003 -0400, Russ White wrote:

> > Hadn't we had agreed that the existing terms would have references added
> > and move to the "Existing Terminology" section?  4 of the 9 terms
> > provided are from John Moy's OSPFv2 RFC.  Those are now referenced
> > (weren't in the previous version), but the still need to go in "Existing
> > Terminology".
>
>We had agreed to put in references, not to remove them from the draft.

"Existing Terminology" is a section in all BMWG Terminology drafts.  This 
draft should also have it.

>The
>references are there. The actual discussions were pulled, but there is
>discussion there that is not in the original reference points. Where else
>would you put those discussions? In an "odd things we couldn't figure out
>what to do with" section of the methdology draft, since they aren't
>methodology? Or in the "things that don't really apply to applicability"
>section of the applicability draft?

You could create a Discussion section - "Discussion - Applying OSPF 
standard terms to OSPF Control Plan Convergence Benchmarking".


> > Hadn't we agreed that defining Internal Measurement, External
> > Measurement, and Multi-Device Measurements in this draft was
> > inappropriate and would be removed?  We did because these are broad terms
> > with a broad scope to the BMWG.  That now covers 7 of the 9 terms.
>
>I don't recall such an agreement. As I recall, we had concluded that they
>are not covered anyplace else, and they are referred to in the other
>drafts, unless you can find some other place where they are defined.

These terms have been used for years in BMWG documents.  It seems 
inappropriate to define the terms in a narrowly focused "Terminology for 
Benchmarking OSPF Control Plane Convergence" document.


> > Hadn't we agreed that the term "Incremental-SPF" would not be included
> > because it is implementation specific.  Cisco also does PRC.  Why not
> > include that?  Avici does RTC.  Let's include that.  Of course this would
> > go on and on so we decided to not include Incremental-SPF.  That is 8 of
> > the 9.
>
>Not that I recall. iSPF is used as an example, in one of the drafts, about
>how implementation type stuff can impact the performance numbers obtained.

Exactly.  You use it in discussion to describe white-box 
implementation.  The term is not required to perform black-box benchmarking.

>In your opinion, it seems we need to either:
>
>-- remove this section of the impacted draft, since we can't define the
>term without defining every instance of implementation level details, and
>we can't include the section unless we define the terms
>
>-- remove the single example, and leave a single terse sentence in the
>draft without any examples
>
>-- add every known or possible example, to be fair
>
>I find all of these alternatives unacceptable, since, again, iSPF is a well
>known concept,

I have been trying to emphasize this point without mentioning vendors.  I 
now have to mention vendors, so I apologize to anyone who prefers 
mailing-list discussion remain vendor-agnostic (as I certainly do).  Recent 
NANOG presentations from Juniper, Cisco, and Avici indicate that only Cisco 
of those 3 implements incremental-SPF.  Since Avici and Juniper have faster 
convergence times then Cisco, Incremental-SPF is clearly not an 
implementation requirement.  The drafts could talk about Incremental-SPF, 
but it has nothing to do with benchmarking convergence.  I like the option 
of not discussing any vendor's implementation for convergence in IETF 
documents.

>but not defined any other place that I can find (in an rfc),
>and we refer to it as an example.
>
> > The 9th term is "Shortest Path First Time".  Mitchell had posted for the
> > previous version the comment "ADD... Some implementations may force a
> > interval between successive SPF calculations. If it exists, then this
> > additional time value should be subtracted from the total time.".  This
> > is a great recommendation to add here in the Terminology draft.  In fact,
> > some implementations have an SPF Delay and an SPF Holdtime.  The
> > definition should state explicitly that measurement includes the SPF
> > Delay, but the SPF HoldTime may be subtracted.  I also recommend that the
> > term be changed to "Shortest Path First Execution Time".  That would then
> > be the only remaining term.
>
>We can add and subtract here. I don't have that email on my list of things
>to change, don't know how I missed it.
>
> > There is one other new term introduced in the Terminology Document,
> > Single Router Convergence (or SR-Convergence).  This definition should be
> > added to the "Terminology" section as the second term rather than in the
> > "Concepts"  section where it is currently located.  I also recommend that
> > based upon the other ongoing work for Data Plane Convergence that the
> > "SR-Convergence"  term be changed to "Control Plane Convergence" to
> > eliminate confusion and match the way most people already speak.  Also,
> > in the sentence "convergence will mean the point in time when the DUT has
> > performed all actions needed to react to the change in topology
> > represented by the test condition", it is recommended to change the word
> > "all" to "some" because the Tree Building and FIB Update are not included
> > in this measurement term.
>
>How does it not include tree building, if we're talking about spf times?
>Isn't the definition of spf building the tree? I don't mind changing the
>section it's in, and I don't care what we call the term. The term was
>originally "invented" to get around the problem that a few people brought
>up in calling what a single router does convergence. We were told we
>couldn't call what happens on a single router "convergence," since
>convergence, as a term, is applicable only to networks (regardless of the
>dictionary definition of the term).

I think "Control Plane Convergence" makes all sides happy. It is also the 
way people speak.


>If people aren't going to raise the "convergence only happens in networks"
>argument again, I'll be glad to do a search and replace, killing off
>SR-convergence, and replacing it with convergence. As for whether you want
>to add "control plane," I think that once we add control plane to SR, then
>we wind up with:

We do need to differentiate Data Plane Convergence in a single router from 
Control Plane Convergence in a single router.  Network Convergence refers 
to network convergence.  Data Plane Convergence and Control Plane 
Convergence refer to a single router.


>CPSR-C
>DPSR-P
>TSP-C
>CPN-C
>DPN-C
>TN-C
>
>Do we really want to go one defining different terms for every aspect of
>convergence we meet, or can we let the context sway the definition? Since
>the drafts are about OSPF,

This raises another issue.  The Methodology draft applies to only OSPF 
intra-Area routes.  What about Inter-Area routes?  What about a mix of 
intra-area and inter-area routes in the route table, which is most often 
the case?  I am afraid the a methodology for only intra-area routes will 
not be of much use.  Did you plan to do additional methodologies for 
inter-area and a mix?

>I would normally take it that it only applies to
>the control plane, within the context. I can see the usefulness in email,
>if no context is set, but context is clearly set in these drafts, and I see
>no reason not to just go with "convergence," without all the additions that
>make it hard to read, just for technical precision.

Of course, "convergence" cannot be the term applied to this work.  It is 
far too general.

Scott


>Russ
>
>__________________________________
>[e-mail] CCIE <>< Grace Alone


_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg