[ippm] Re: [bmwg] Re: Introducing: AI Fabric Benchmarking Methodology Suite (Training, Inference, Terminology)

[kehan yao <[e-mail]>]

Hi Med, all,

Just a clarification question.

Can IETF standardize or publish benchmarking methodologies specifications
which describe techniques designed outside of IETF? For example, RoCEv2,
UET, etc.

Overall, I think the document is very useful and hit hot topics.

Best regards,
Kehan

<[e-mail]> 于2026年3月3日周二 23:50写道：

> Hi Fernando,
>
>
>
> Thank you for sharing this.
>
>
>
> I had troubles to find the drafts by the names indicated below. I guess
> the correct files are the following:
>
>
>
>    -
>    https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-terminology/
>    -
>    https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-training-bench/
>    -
>    https://datatracker.ietf.org/doc/draft-calabria-bmwg-ai-fabric-inference-bench/
>
>
>
> Cheers,
>
> Med
>
>
>
> *De :* Fernando Calabria (fcalabri) <fcalabri=[e-mail]>
> *Envoyé :* vendredi 27 février 2026 15:14
> *À :* [e-mail]
> *Objet :* [bmwg] Introducing: AI Fabric Benchmarking Methodology Suite
> (Training, Inference, Terminology)
>
>
>
>
>
> Dear BMWG Participants,
>
>
>
> We are pleased to introduce three companion individual submissions
> addressing a benchmarking methodology gap for AI/ML network fabrics:
>
>
>
>   (1) draft-calabria-pignataro-bmwg-ai-fabric-terms-00
>
>       Terminology for AI Fabric Benchmarking
>
>
>
>   (2) draft-calabria-pignataro-bmwg-ai-fabric-training-bench-00
>
>       Benchmarking Methodology for AI Training Fabric Networks
>
>
>
>   (3) draft-calabria-pignataro-bmwg-ai-fabric-inference-bench-00
>
>       Benchmarking Methodology for AI Inference Fabric Networks
>
>
>
> WHY AI FABRICS REQUIRE NEW BMWG WORK
>
>
>
> AI/ML workloads impose network behaviors with no analog in existing BMWG
> methodology documents:
>
>
>
>   Training: Bulk synchronous collective operations (AllReduce, AllGather,
> ReduceScatter) over RoCEv2 fabrics create synchronized incast-like traffic
> bursts, making tail latency and congestion management (PFC/ECN/DCQCN) the
> dominant performance determinants — not throughput or packet loss in
> isolation.
>
>
>
>   Inference: Disaggregated prefill/decode architectures and
> Mixture-of-Experts (MoE) routing generate highly asymmetric, bursty
> point-to-point KV cache transfer patterns. SLA-relevant KPIs are Time to
> First Token (TTFT) and Inter-Token Latency (ITL), which are jointly
> determined by compute and fabric behavior in ways that require careful test
> isolation methodology.
>
>
>
> DOCUMENT STRUCTURE
>
>
>
> The three documents follow established BMWG convention: a terminology
> companion paired with separate methodology documents for the two principal
> AI workload classes. All documents are scoped to controlled laboratory
> environments, maintain strict vendor neutrality, and define no acceptance
> thresholds.
>
>
>
> We welcome Working Group review and would appreciate Chair guidance on
> presentation opportunities.
>
> We are also interested in coordinating with authors of
> draft-gaikwad-llm-benchmarking-methodology and any related BMWG efforts.
>
>
>
> Thank you for your time and consideration.
>
>
>
> Fernando Calabria (Cisco) ,  Carlos Pignataro (Blue Fern Consulting) , Giuseppe
>  Fioccola and Qin Wu (Huawei)
>
>
>
> ____________________________________________________________________________________________________________
> Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
> Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and delete this message and its attachments.
> As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
> Thank you.
>
> _______________________________________________
> bmwg mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>