RE: [bmwg] WLAN switch controller I-Ds for BMWG to consider

["Tom Alexander" <[e-mail]>]

David,

Thanks for your support, and for your detailed review and feedback. Let me
address each of your points in turn.

Item 1. Generally agreed. I've found several vendors (no names, please! ;-)
playing games with MAC protocol parameters to gain preferential access to
the medium. I was trying to avoid the notion of running a functional test on
the SUT, but you are quite correct that we need some way of either detecting
or, at a minimum, reporting these games. (If anyone on this list can point
us at an I-D or RFC to serve as an example, that would be great.)

Item 2. Also generally agreed. This is an area that has to be cleaned up.
There is a distinction between "open-air testing" (which should be heavily
discouraged or outlawed for serious benchmarking efforts) versus
"over-the-air testing" (which is done in calibrated anechoic chambers and is
therefore both predictable and reproducible). The document fuzzes over the
two. This needs to be fixed.

Item 3. Thanks for pointing this out - the next revision will make this
change.

Item 4. I confess that my professorial instinct to say "... and this is left
as an exercise for the reader" is showing through. :-) I will clean up all
the casual "etc"s in the next revision.

Again, I appreciate the review.

Best regards,

- Tom

-----Original Message-----
From: David Newman [mailto:[e-mail]] 
Sent: Tuesday, July 10, 2007 9:33 PM
To: [e-mail]
Cc: 'Muninder Sambi (msambi)'; 'Jerry Perser'
Subject: Re: [bmwg] WLAN switch controller I-Ds for BMWG to consider

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 6/8/07 1:49 PM, Tom Alexander wrote:
> BMWGers,
> 
> Back at the 67th IETF last November, there was a presentation discussing a
> proposal on CAPWAP (WLAN) switch controller benchmarking. The presentation
> ended by threatening to solicit some help and submit some drafts.
> 
> The help was solicited, and two drafts have been submitted to the IETF I-D
> database:
> 
> draft-alexander-bmwg-wlan-switch-term-00.txt  -- terminology
> draft-alexander-bmwg-wlan-switch-meth-00.txt -- methodology
> 
> Review and comments on these drafts would be much appreciated.

I support the bmwg undertaking these IDs as work items, and congratulate
the authors for posting strong draft zeros.

There are several areas where I believe more specifics would help:

1. In any environment where contention for the medium exists -- and
802.11 WLANs are a poster child for this -- the device that puts a frame
on the medium first wins. This has led to a sort of arms race among
vendors for control of the airwaves.

The methodology makes no mention of three contention mechanisms that can
heavily influence test results. In my experience these mechanisms have
been the main factors in differentiating performance from different
vendors' systems and thus should be included in any test methodology:

The mechanisms are:

a. Interframe space type. It is possible and standards-compliant to use
SIFS instead of DIFS interframe space) under some circumstances. At a
minimum, the type of IFS in use should be declared. Better still would
be a test to measure the IFS in use.

Early Ethernet switches cheated on IFS to gain an advantage in
half-duplex environments. While I am not aware of IFS cheating in WLAN
equipment, it is possible and desirable to measure the spaces between
frames.

b. Coordination function type. Some WLAN access points implement PCF
(point coordination function) instead of the more common DCF
(distributed coordination function). This can be highly desirable in
situations where the flow of traffic is primarily downstream from WTP to
end-station. It gives the WTP far more opportunities for access to the
medium than end-stations.

Again, the CF type in use should be declared at a minimum. More
preferable would some externally observed measurement of the CF type.

c. Client contention behavior. Most but not all WLAN stations use a
exponential backoff algorithm to deal with contention for the medium.
Obviously, a end-station or WTP that fudges this a bit will get its
frame on the medium first, potentially "winning" every collision battle.
(This is a crude example; more sophisticated variations of this might
involve winning only some percentage of the time and/or altering the
winning percentage in response to RF or other conditions.)

It is possible and desirable to measure client contention behavior. I
encourage development of tests in this area.

2. In section 3.3.3.1 of the test methodology ID, the description of the
open-air test bed is seriously underspecified, which almost certainly
will lead to a lack of reproducibility of test results in different labs.

Rather than adding specificity, I would rather see open-air testing
discouraged altogether in the context of this document. (Note: I have
nothing against open-air testing in a *network* context, specific to one
operators' network; in a *lab* context, however, there's just too much
variability.)

3. Just a wording nit in both documents:

s/best efforts/best effort/

4. Also a wording nit, but again in both documents there are several
instances of the term "etc." Specific enumeration would be better.

dn

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFGlF2AyPxGVjntI4IRAnWOAKC3u/oHgUenovoMERdpvS88e0u8gwCdEOGY
2VKvbR7qDqCk+lLRovJyFIw=
=uzNP
-----END PGP SIGNATURE-----

_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg


_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg