Re: [bmwg] AD Eval of draft-ietf-bmwg-vswitch-opnfv

[Warren Kumari <[e-mail]>]

Thank you.

IETL LC started.

W

On Mon, May 1, 2017 at 1:08 PM, MORTON, ALFRED C (AL) <[e-mail]> wrote:
> Hi Warren,
>
> The latest version of this draft:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-vswitch-opnfv/
>
> addresses your comments plus some other editorial
> clean-up, and includes an abbreviations section
> (which should help with further reviews).
>
> thanks again for your review,
> Al (for the co-authors)
>
>> -----Original Message-----
>> From: MORTON, ALFRED C (AL)
>> Sent: Tuesday, April 25, 2017 7:43 AM
>> To: 'Warren Kumari'; [e-mail];
>> [e-mail]; [e-mail]
>> Subject: RE: AD Eval of draft-ietf-bmwg-vswitch-opnfv
>>
>> Thanks for your review & comments, Warren; the edits will
>> take a few days to reach the top of the ToDo list.
>> Al
>>
>> > -----Original Message-----
>> > From: Warren Kumari [mailto:[e-mail]]
>> > Sent: Sunday, April 23, 2017 10:40 AM
>> > To: [e-mail]; [e-mail];
>> > [e-mail]
>> > Subject: AD Eval of draft-ietf-bmwg-vswitch-opnfv
>> >
>> > Hi there,
>> >
>> > I've just done the AD eval of draft-ietf-bmwg-vswitch-opnfv - thank
>> > you for writing this, I think that it is a useful document.
>> >
>> > I had a few questions and comments which I think  it would be useful
>> > to address before starting IETF LC. I'm not a benchmarking (or NFV /
>> > vswitch person), so some of these questions / comments may be very
>> > basic.
>> >
>> > I've noted my questions with [WK: ...]. My questions / comments look
>> > long, because I kept much of the text to make readability easier.
>> >
>> > ---------
>> >
>> >
>> > 2.  Scope
>> >
>> >    The primary purpose and scope of the memo is to inform the industry
>> >    of work-in-progress that builds on the body of extensive BMWG
>> >    literature and experience, and describe the extensions needed for
>> >    benchmarking virtual switches.  Inital feedback indicates that many
>> >
>> > [WK: s/Inital/Initial/]
>> >
>> >    of these extensions may be applicable beyond the current scope (to
>> >    hardware switches in the NFV Infrastructure and to virtual routers,
>> >    for example).  Additionally, this memo serves as a vehicle to
>> include
>> >    more detail and commentary from BMWG and other Open Source
>> >    communities, under BMWG's chartered work to characterize the NFV
>> >
>> >    Infrastructure (a virtual switch is an important aspect of that
>> >    infrastructure).
>> >
>> >    The benchmarking covered in this memo should be applicable to many
>> >    types of vswitches, and remain vswitch-agnostic to great degree.
>> >
>> > [WK: Please expand / define vswitch - perhaps in the parenthesis
>> above.]
>> >
>> >    There has been no attempt to track and test all features of any
>> >    specific vswitch implementation.
>> >
>> > ...
>> >
>> > 3.1.  Comparison with Physical Network Functions
>> >
>> >    To compare the performance of virtual designs and implementations
>> >    with their physical counterparts, identical benchmarks are needed.
>> >    BMWG has developed specifications for many network functions this
>> >    memo re-uses existing benchmarks through references, and expands
>> them
>> >    during development of new methods.
>> >
>> > [ WK: This sentence makes no sense / is missing some words / should be
>> > 2 sentences / something....]
>> >
>> >    A key configuration aspect is the
>> >    number of parallel cores required to achieve comparable performance
>> >    with a given physical device, or whether some limit of scale was
>> >    reached before the cores could achieve the comparable level.
>> >
>> >    It's unlikely that the virtual switch will be the only application
>> >    running on the SUT, so CPU utilization, Cache utilization, and
>> Memory
>> >    footprint should also be recorded for the virtual implementations
>> of
>> >    internetworking functions.
>> >
>> > ...
>> >
>> > 3.3.  New Configuration Parameters
>> >
>> >    A key consideration when conducting any sort of benchmark is trying
>> >    to ensure the consistency and repeatability of test results.  When
>> >    benchmarking the performance of a vSwitch
>> >
>> > [ WK: Please choose a single term (vswitch / vSwitch) ]
>> >
>> >    there are many factors that
>> >    can affect the consistency of results, one key factor is matching
>> the
>> >    various hardware and software details of the SUT.  This section
>> lists
>> >    some of the many new parameters which this project believes are
>> >    critical to report in order to achieve repeatability.
>> >
>> >    Hardware details including:
>> >
>> >    o  Platform details
>> >
>> >    o  Processor details
>> >
>> >    o  Memory information (type and size)
>> >
>> > [ WK: I would also think that the memory layout is important - if not,
>> > ignore. Oh, it is mentioned below ("Memory DIMM configurations"). This
>> > would be easier to read if you grouped memory together, processor /
>> > cores together, etc.]
>> >
>> >
>> >    o  Number of enabled cores
>> >
>> >    o  Number of cores used for the test
>> >
>> >    o  Number of physical NICs, as well as their details (manufacturer,
>> >       versions, type and the PCI slot they are plugged into)
>> >
>> >    o  NIC interrupt configuration
>> >
>> > [WK: What about other NIC features (DDP, checksum magic,
>> > scatter-gather, etc)? Or are these not applicable? (if not, ignore) ]
>> >
>> >    o  BIOS version, release date and any configurations that were
>> >       modified
>> >
>> >    o  CPU microcode level
>> >
>> >    o  Memory DIMM configurations (quad rank performance may not be the
>> >       same as dual rank) in size, freq and slot locations
>> >
>> >    o  PCI configuration parameters (payload size, early ack option...)
>> >
>> >    o  Power management at all levels (ACPI sleep states, processor
>> >       package, OS...)
>> >
>> >    Software details including:
>> >
>> >    o  OS parameters and behavior (text vs graphical no one typing at
>> the
>> >       console on one system)
>> >
>> >    o  OS version (for host and VNF)
>> >
>> >    o  Kernel version (for host and VNF)
>> >
>> >    o  GRUB boot parameters (for host and VNF)
>> >
>> >    o  Hypervisor details (Type and version)
>> >
>> >    o  Selected vSwitch, version number or commit id used
>> >
>> >    o  vSwitch launch command line if it has been parameterised
>> >
>> >    o  Memory allocation to the vSwitch
>> >
>> >    o  which NUMA node it is using, and how many memory channels
>> >
>> >    o  DPDK or any other SW dependency version number or commit id used
>> >
>> >    o  Memory allocation to a VM - if it's from Hugpages/elsewhere
>> >
>> > [WK: Hugpages? It this the emo version of hugepages? :-) ]
>> >
>> >
>> >    o  VM storage type: snapshot/independent persistent/independent
>> non-
>> >       persistent
>> >
>> >    o  Number of VMs
>> >
>> >    o  Number of Virtual NICs (vNICs), versions, type and driver
>> >
>> >
>> > [ WK: What about IO virtualization stuff like SR-IOV and similar. Or
>> > not applicable?]
>> >
>> >    o  Number of virtual CPUs and their core affinity on the host
>> >
>> >    o  Number vNIC interrupt configuration
>> >
>> >    o  Thread affinitization for the applications (including the
>> vSwitch
>> >       itself) on the host
>> >
>> >    o  Details of Resource isolation, such as CPUs designated for Host/
>> >       Kernel (isolcpu) and CPUs designated for specific processes
>> >       (taskset). - Test duration. - Number of flows.
>> >
>> >    Test Traffic Information:
>> >
>> >    o  Traffic type - UDP, TCP, IMIX / Other
>> >
>> >
>> > [ WK: I don't think that IMIX is a well known acronym. I think either
>> > define it (or ref: RFC6985?), or drop it (the section is "some of the
>> > many new parameters"...)]
>> >
>> >    o  Packet Sizes
>> >
>> >    o  Deployment Scenario
>> >
>> > 3.4.  Flow classification
>> >
>> >    Virtual switches group packets into flows by processing and
>> matching
>> >    particular packet or frame header information, or by matching
>> packets
>> >    based on the input ports.  Thus a flow can be thought of a sequence
>> >    of packets that have the same set of header field values (5-tuple)
>> or
>> >    have arrived on the same port.  Performance results can vary based
>> on
>> >    the parameters the vSwitch uses to match for a flow.  The
>> recommended
>> >    flow classification parameters for any vSwitch performance tests
>> are:
>> >    the input port, the source IP address, the destination IP address
>> and
>> >    the Ethernet protocol type field.
>> >
>> > [ WK: Is it useful to say "5-tuple" above? To me 5-tuple is src ip,
>> > src port, dst ip, dst port, ip protocol. This says for vswitch it is
>> > input port, src ip, dst ip and eth protocol. The two close together
>> > may confuse readers - I think just drop "(5-tuple)". Also, this says
>> > "input port" - is that actually input interface / should it be called
>> > "physical port"? (when hearing IP and then port, people think L3
>> > port)]
>> >
>> >
>> >    o  Scalability Tests to understand how the virtual switch performs
>> as
>> >       the number of flows, active ports, complexity of the forwarding
>> >       logic's configuration... it has to deal with increases.
>> >
>> > [ WK: Looks like got sidetracked in this sentence. ]
>> >
>> >
>> > ---------
>> >
>> > Thanks again,
>> > W
>> >
>> > --
>> > I don't think the execution is relevant when it was obviously a bad
>> > idea in the first place.
>> > This is like putting rabid weasels in your pants, and later expressing
>> > regret at having chosen those particular rabid weasels and that pair
>> > of pants.
>> >    ---maf



-- 
I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.
   ---maf