Feedback on bmwg-lanswitch-03 draft.

[Kevin Dubray <[e-mail]>]

Bob,

It appears that your draft has undergone quite an evolution over the
last two revisions.  Here is my input on your latest version,
draft-ietf-bmwg-lanswitch-03.txt.

2.4 Meshed traffic

In my mind, it looks as you defined the concept of "meshed
traffic" to mean "fully" meshed or what you originally proposed in
earlier drafts as "multidirectional."  And this appears consistent
with the minutes of the last session.

In retrospect, this may be an unduly restrictive definition.  Indeed,
section 16 of RFC 1944 (i.e., Multi-port) refers to a meshed distribution 
of traffic, albeit not a fully meshed distribution. (Scott, correct
me if I'm wrong, but isn't the addressing suggested in RFC 1944, 
sec. 16 was what you were referring to when trying to offered the
term "meshed?")

Regardless, both "partial" and "full" mesh traffic paradigms
are important, valid traffic distribution patterns in testing.
So perhaps a base definition, followed by appropriate extensions
are in order.  For instance:

   2.4 Meshed traffic:  A traffic distribution pattern whereby 
       multiple frames offered to an ingress port of a System
       Under Test are destined to two or more ports of that system.

       Discussion:  In other words, frames offered to a port of a 
       SUT in a meshed traffic distribution are not all destined to the
       same egress port of that SUT.

   2.4.1 Partially meshed traffic:  A type of meshed traffic
       distribution whereby multiple frames offered to an ingress
       port of a SUT are destined to a set of egress ports of that
       SUT.

   2.4.2 Fully meshed traffic:  A type of meshed traffic distribution
       whereby multiple frames offered to an ingress port of a SUT
       are destined to all other tested ports of the SUT with the
       exception of that ingress port itself.

   2.x Non-meshed traffic:  A traffic distribution whereby frames offered
       on a ingress port of a SUT are destined for a single egress
       port of the SUT in a one-to-one mapping.
       

I think the above discussion is "backwards compatible" with historical 
references to "mesh" and "multi-port" but fully captures 
what you are trying to communicate with "multidirectional."  

As opposed to its use as a absolute benchmark, the power of the employing
the "full-meshed" or "multidirectional" traffic model is its diagnostic
capability.  The use of the full meshed traffic model can suggest that
additionally inspection of the SUT may be warranted.  This idea has been
offered at several BMWG sessions, but it has failed to be preserved in the
current discussion.  

==================================================================

2.9 Offered Load 

The definition of offered load says "..the number of frames per second
that an external source attempts to transmit..."  Later in the discussion,
you suggest a more rigorous criterion by stating, "Frames which are not
successfully transmitted by an external source ... MUST NOT be counted as
as transmitted frames..."  In my mind, your last statement contradicts
the formal definition.  If I set the transmit knob of the test tool to
14,880pps and the actual transmission rate is 14,773pps, which value
do I report for offered load?  The tool _attempted_ to transmit at 14,880,
but could, for whatever reason, muster only 14,773.

I would offer the suggestion to modify the wording of "attempts to transmit or
address" to "successfully transmits."  It is very important to absolutely
describe the external stimuli presented to the SUT for action.

You may wish to consult RFC 1242 and 1944 in light of your last paragraph
in this section.  The implication of suppressing the propagation of BPDUs or
other like mechanisms may be counter to these documents, if my memory serves
me correctly. 


2.8 Port Load

I think I know what you are trying to convey here, (i.e., define load with
respect to port I/O), but I'm not so sure that I agree with your definition.  
The sticking point for me is where you say the port load is the number of fps 
that a DUT's port can be "observed to transmit and receive in response to an 
offered load." 

Ascertaining a tested port's ability to transmit frames is usually
straightforward enough. I'm guessing that you're assuming the correct 
forwarding of a frame implies its successful reception by the test tool. 
No argument there.  Without intrusive methods - that is directly relying on 
the DUT's statistics - determining whether a port "received" is not so 
straightforward.

Consider the case where a frame offered to a DUT is actually read off the
medium by the DUT's port but dropped internally by the DUT (e.g. clipped
across the "switch fabric").  What signifies that the frame was correctly 
"received"? I think what you are really striving to communicate is the 
observed utilization of the medium attached to the port while taring any 
non-test specific traffic components from the measurement.  

2.10 Balanced Load

Again, I understand the concept because I am cognizant of the test condition
that you are trying to describe.  I've distributed the definition around
to those both familiar with network benchmarking and those who are not.  All
suggest this definition should be made more clear.  



With regards to this discussion on types of loads, I noticed you 
dropped the concept of intended load that was offered in version 02 of your 
draft.  While "intended load" doesn't appear crucial in your discussion 
of switches per se, I maintain (and the last BMWG minutes reflect) that it is 
important to differentiate the load the test technician asks the test tool
to deliver from the observed load the test tool actually offers.  The
definition of intended load may coax the test technician to consider the
following heuristic:

       if (offered load NOT_EQUAL intended load)
           then understand_why;

Whether the reason why offered load differs from intended load is collisions,
flow control, or the fact that perhaps the test tool just doesn't deliver the 
intended load, it is a important data point in the interpretation of the
results. Furthermore, it's a datapoint that not all test technicians may
consider scrutinizing up front. I would strongly advocate reintroducing the 
concept of intended load back into your draft.


2.12 Overload.

In the definition section, you state overload as the condition where
port(s) are loaded "in excess of the maximum rate of transmission allowed
by the medium." Yet in the discussion section, you state "Port overloading... 
requires a minimum of two input ports and one output ports..." Doesn't the
case where a test tool presents a DUT port with sub-minimal (illegal) 
IFG-spaced frames (in the case of Ethernet) fulfill your definition's 
requirement for overload?

RFC 1242 presents overloaded behavior (section 3.12) as a systemic DUT 
condition; your definition presents a physical medium context.

I think your definition may need to be reworked to reflect the spirit of
your discussion. Or vice versa.


=============================================================================
2.13 Forwarding Rate

In the presented discussion, use of the term "throughput" may be ambiguous
in light of the definition of throughput as stated in section 3.17 of RFC
1242. 

Perhaps replacing "throughput" with "frame forwarding" may be more 
consistent?


2.14 Forwarding Rate at Maximum Load.

It is not that uncommon to find test tools that CANNOT generate Maximum Load,
as you previously define.  Moreover, "Maximum Load" may vary with respect
to other parameters other than a medium's theoretical, maximum utilization.
For example, those media employing tokens, "maximum load" may vary as a 
function of Token Rotation Time, Token Holding Time, or the ability to chain
multiple frames to a single token, etc.  Recall that the last session
of the BMWG suggested that the term "Forwarding Rate at Maximum Offered Load"
(FRMOL) be defined.  In the case where the max theoretical utilization of the 
medium can be generated by the test tool, max offered load IS maximum load 
(as you define). In the case where "maximum load" cannot be generated by the 
test tool, maximum offered load represents the upper (max) test bound. My
counsel would be to reflect the BMWG's suggestion. 

The group also suggested defining "Maximum Forwarding Rate" (MFR) to be the
largest forwarding rate from a set of forwarding rate measurements.  It was
further offered that comparison between MFR and FRMOL was useful in 
characterizing DUT behavior.   


=============================================================================
2.16 Forward pressure

While the current definition is adequate in describing a
technique of violating Ethernet's backoff algorithm, perhaps it
could be expanded slightly to reflect an increased scope of a 
network condition to be identified:

    Forward Pressure: A behavior whereby the SUT applies a heuristic
    that departs from or otherwise violates a defined, standardized
    protocol as a tradeoff for increased forwarding performance in
    those network conditions where congestion or contention is 
    present.

This definition would allow for the identification of the disregard
of backoff algorithms, flow control primitives, or other medium
capture constraints.  Wording should be offered to cite that such behavior
is unacceptable.  


=============================================================================
2.18 Address handling

Perhaps changing the term "address handling" to "address handling capacity"
or something similar better conveys what it is that you are trying to describe?


=============================================================================
2.20 Flooding

You seem to define a network condition here as opposed to a benchmark. 
If "flooding" isn't a benchmark, are the "Measurement units" applicable 
and/or necessary?


2.21 Illegal frames.

Again this appears to be a term where you define a specific condition, that
is, when frames are to be considered poorly formed with respect to the PHY/MAC
layer.  Being that the term resembles more a definition than benchmark, does 
the specification of measurement units make sense here?

============================================================================

Miscellaneous.

Section 3, the Index of Definitions, is wonderful.  However, it would have
been my personal preference to present it earlier to the reader.

Best Regards,
Kevin