[bmwg] Packet generator precision

Paul Emmerich <emmericp@net.in.tum.de> Thu, 14 June 2018 20:11 UTC

Return-Path: <emmericp@net.in.tum.de>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3E349130EE0 for <bmwg@ietfa.amsl.com>; Thu, 14 Jun 2018 13:11:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.199
X-Spam-Level:
X-Spam-Status: No, score=-4.199 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PoltyZIM4bL5 for <bmwg@ietfa.amsl.com>; Thu, 14 Jun 2018 13:11:22 -0700 (PDT)
Received: from mail-out1.informatik.tu-muenchen.de (mail-out1.informatik.tu-muenchen.de [131.159.0.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 95D77130E98 for <bmwg@ietf.org>; Thu, 14 Jun 2018 13:11:22 -0700 (PDT)
Received: from p200300e9cbd34068adcaac1a0060b937.dip0.t-ipconnect.de (p200300E9CBD34068ADCAAC1A0060B937.dip0.t-ipconnect.de [IPv6:2003:e9:cbd3:4068:adca:ac1a:60:b937]) (Authenticated sender: emmericp) by mail.net.in.tum.de (Postfix) with ESMTPSA id A651C2895021 for <bmwg@ietf.org>; Thu, 14 Jun 2018 22:11:14 +0200 (CEST)
From: Paul Emmerich <emmericp@net.in.tum.de>
X-Pgp-Agent: GPGMail
Content-Type: multipart/signed; boundary="Apple-Mail=_851CEBC5-80D1-4337-88A8-815C1F52641D"; protocol="application/pgp-signature"; micalg="pgp-sha512"
Date: Thu, 14 Jun 2018 22:11:12 +0200
Message-Id: <A0131167-1BB5-4BAE-95F3-D5265647454E@net.in.tum.de>
To: bmwg@ietf.org
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
X-Mailer: Apple Mail (2.3124)
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/1UJUpn7EBXAvHMlihJoNpMiurf8>
Subject: [bmwg] Packet generator precision
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 14 Jun 2018 20:11:25 -0000

Hi,

we've had a (very) short discussion on reliability and precision of software packet generators at the session at IETF 100.

I've wanted to revive this discussion because I might be at IETF 102 (travel grant application pending, fingers crossed),
if anyone here is still interested in this topic.

To get the discussion started, here's our research paper from last year where we looked at rate control of packet generators:
https://www.net.in.tum.de/fileadmin/bibtex/publications/papers/comparison_of_software_packet_generators.pdf

I want to draw your attention to Figure 1 on page 2. It shows a measurement of the median latency of packet forwarding with Open vSwitch
under increasing load. We repeated this test several times and modified the burst size of the packet generator while keeping the average
packet rate the same. The base line is true CBR traffic.

The result is an increase of the measured latency of up to 400% even though we only changed the spacing between packets on the wire.

Why is this important? Packet generators often generate small bursts in order to increase performance.
Usually the default is bursts of 16 to 256 packets and that can ruin latency tests in some cases.

The short discussion at IETF 100 was whether it might be helpful to have some kind of test for packet generators to measure their precision.


Paul


--
Chair of Network Architectures and Services
Department of Informatics
Technical University of Munich
Boltzmannstr. 3
85748 Garching bei München, Germany