[ippm] Re: [bmwg] Introducing: AI Fabric Benchmarking Methodology Suite (Training, Inference, Terminology)

[[e-mail]]

Hi Fernando,

Thank you for sharing this.

I had troubles to find the drafts by the names indicated below. I guess the correct files are the following:


  *   https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-terminology/
  *   https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-training-bench/
  *   https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-inference-bench/

Cheers,
Med

De : Fernando Calabria (fcalabri) <fcalabri=[e-mail]>
Envoyé : vendredi 27 février 2026 15:14
À : [e-mail]
Objet : [bmwg] Introducing: AI Fabric Benchmarking Methodology Suite (Training, Inference, Terminology)



Dear BMWG Participants,

We are pleased to introduce three companion individual submissions addressing a benchmarking methodology gap for AI/ML network fabrics:

  (1) draft-calabria-pignataro-bmwg-ai-fabric-terms-00
      Terminology for AI Fabric Benchmarking

  (2) draft-calabria-pignataro-bmwg-ai-fabric-training-bench-00
      Benchmarking Methodology for AI Training Fabric Networks

  (3) draft-calabria-pignataro-bmwg-ai-fabric-inference-bench-00
      Benchmarking Methodology for AI Inference Fabric Networks

WHY AI FABRICS REQUIRE NEW BMWG WORK

AI/ML workloads impose network behaviors with no analog in existing BMWG methodology documents:

  Training: Bulk synchronous collective operations (AllReduce, AllGather, ReduceScatter) over RoCEv2 fabrics create synchronized incast-like traffic bursts, making tail latency and congestion management (PFC/ECN/DCQCN) the dominant performance determinants - not throughput or packet loss in isolation.

  Inference: Disaggregated prefill/decode architectures and Mixture-of-Experts (MoE) routing generate highly asymmetric, bursty point-to-point KV cache transfer patterns. SLA-relevant KPIs are Time to First Token (TTFT) and Inter-Token Latency (ITL), which are jointly determined by compute and fabric behavior in ways that require careful test isolation methodology.

DOCUMENT STRUCTURE

The three documents follow established BMWG convention: a terminology companion paired with separate methodology documents for the two principal AI workload classes. All documents are scoped to controlled laboratory environments, maintain strict vendor neutrality, and define no acceptance thresholds.

We welcome Working Group review and would appreciate Chair guidance on presentation opportunities.
We are also interested in coordinating with authors of draft-gaikwad-llm-benchmarking-methodology and any related BMWG efforts.

Thank you for your time and consideration.


Fernando Calabria (Cisco) ,  Carlos Pignataro (Blue Fern Consulting) , Giuseppe  Fioccola and Qin Wu (Huawei)

____________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.