Re: [tcpm] A description of Linux pacing

Yuchung Cheng <ycheng@google.com> Wed, 20 March 2024 18:56 UTC

Return-Path: <ycheng@google.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99778C14F6A2 for <tcpm@ietfa.amsl.com>; Wed, 20 Mar 2024 11:56:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.606
X-Spam-Level:
X-Spam-Status: No, score=-17.606 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kanZeVb-19Sd for <tcpm@ietfa.amsl.com>; Wed, 20 Mar 2024 11:56:39 -0700 (PDT)
Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DF874C14F69E for <tcpm@ietf.org>; Wed, 20 Mar 2024 11:56:39 -0700 (PDT)
Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-a4644bde1d4so21816666b.3 for <tcpm@ietf.org>; Wed, 20 Mar 2024 11:56:39 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1710960998; x=1711565798; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=f3COD/zGspy1Pn0YIYLEosj6qvysnyZIm4qefRhXF3M=; b=yFOyDqECyyR9gPNSjmyQV59FH0FbkARbFegZThAr/8KDNNEMMoFt+ALmOtxqpKBkS1 u5cZv7+WZltFneC6Fjq6CVpIXj0yxATy6CwvxTU0m9k8CR7u6Yt8yjCKrtrZ60vorACc /uyxKU30OU1zD/3h0U6HqWgywrZizuZofI0idG480M1eQeGRKWECcI6rZP8MQLTO9c5Z t2lXjDT0fj0w64XXMZYz872wmexN/lU07vDfPPV1/a1RbX5N9goRpATjBYCLncnkBr5I 3sJ2dcYN4uJ/K9xYCAmCeIEaKc7b/WrJrfjWtDprirxTMzNcTKVNFEs9cJbpqN4CeDDe 8tGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710960998; x=1711565798; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f3COD/zGspy1Pn0YIYLEosj6qvysnyZIm4qefRhXF3M=; b=W2ltMp6Yz2wtYCHduDg0dHxkTUvBaqZf6SNh24I6ypZoi7yJl9gXwsbd5Z22h38UZ+ y2QTeSyaaiO0bYDK27/HIxKUUmL4IzlHunOaGUfA/tMIQT0u6f9QZNgs1FF+TQ5oVfQ+ NapB2wHIijzLutSi3orPtjbbzzIaWh2YlubyrZUbZBHLwkni5QaVLzym+HLlUJWlAfzu 2TwEIbk9tpqHCBA9e1hKZeSDUWe4Rw4TVK2K6V+PzVpG89FJ8TYU1BfXtULFCaMSaKfA zoI6xAcTl/F9NKaR2auKEPVmlBQwZKugvRKFOCHGtLrlL4qe8oPTr/1qMR0Rmoev0J6Z 4fUQ==
X-Forwarded-Encrypted: i=1; AJvYcCW5dgGhGHSrJX1vp7JSe/y3BRCGlhCJHXCnI3tx0lsNeoSr9zQ4FBhb44vgIBPsKfy2nayVdC5SR9cLXHbI
X-Gm-Message-State: AOJu0YyDqhfTKtWlTa3aH7rCHWWTk+qpBQBVQQeFbrUHWeMmsabUlfQE +i26f4tJzVqh4ASEKHZTbDLG8PSFa5mF1RtkPbT0pCZ3NrIi1ltr3f+m7EVwiJviprBHnzw+oNS Qo3C2A1kdRGTcdL9i29p+zq6Ykr5MoGWwETi3
X-Google-Smtp-Source: AGHT+IHU8AnMhDieXN0A9R6gCRs2SR/kl+NYkgtJXA+/sxFtHArlrLk/Uv45pgNYIIFbEJuxDvjC895QA2yQg8x1Ysk=
X-Received: by 2002:a17:906:abce:b0:a46:d9a0:ddd7 with SMTP id kq14-20020a170906abce00b00a46d9a0ddd7mr2074187ejb.68.1710960997422; Wed, 20 Mar 2024 11:56:37 -0700 (PDT)
MIME-Version: 1.0
References: <AE83F8F4-7D71-4034-93E7-365E1CD701F5@ifi.uio.no> <CAM4esxQg=MmBMJ-2dyWLX0HCy9yBCCaVBaWof31MFNBC-37ajQ@mail.gmail.com> <16FCCC7E-7F2E-453D-AD9B-BA6942C7B2F9@ifi.uio.no>
In-Reply-To: <16FCCC7E-7F2E-453D-AD9B-BA6942C7B2F9@ifi.uio.no>
From: Yuchung Cheng <ycheng@google.com>
Date: Wed, 20 Mar 2024 11:55:55 -0700
Message-ID: <CAK6E8=e8zKtnKKinu3d81Wpk2gYRa8iPMO+Z9ynSd8PSP0Fp0A@mail.gmail.com>
To: Michael Welzl <michawe@ifi.uio.no>
Cc: Martin Duke <martin.h.duke@gmail.com>, tcpm@ietf.org, iccrg@irtf.org, Eric Dumazet <edumazet@google.com>, Neal Cardwell <ncardwell@google.com>
Content-Type: multipart/alternative; boundary="0000000000007a303606141c26e2"
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/odp0nphv8yCK3yrnge532MK5l98>
Subject: Re: [tcpm] A description of Linux pacing
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Mar 2024 18:56:43 -0000

Neal and I have a short paper on Linux pacing, which integrates TSQ, and
TSO autosizing. it has evolved over the years but the core design remains.
https://netdevconf.info/1.2/papers/bbr-netdev-1.2.new.new.pdf

On Wed, Mar 20, 2024 at 12:42 AM Michael Welzl <michawe@ifi.uio.no> wrote:

> Hi,
>
> and thanks for your feedback!
>
> I agree; I can write this up in time for the Vancouver IETF. As for
> documenting other implementations, I don’t know internals of any others…
> but I’m very willing to involve others in such an I-D - if any expert on a
> different OS is interested, get in touch!
>
> Else, I could just make a start with this and see where it goes. It’s not
> much work, I can keep it short and simple.
>
> Cheers,
> Michael
>
>
> On 12 Mar 2024, at 23:08, Martin Duke <martin.h.duke@gmail.com> wrote:
>
> Speaking as an individual, it's a bummer we don't have a real RFC about
> pacing, this would be a decent Informational RFC for a way to do pacing,
> that is also a useful reference for how an important implementation does it.
>
> Even more ambitious, one could document the approaches of several major
> implementations.
>
> On Mon, Feb 19, 2024 at 6:06 AM Michael Welzl <michawe@ifi.uio.no> wrote:
>
>> Dear TCPM and ICCRG (I assume that there’s so much overlap with people in
>> CCWG that it would just be spamming to send it there too?),
>>
>> Over the last two weeks or so, I have put some effort into trying to
>> understand how Linux pacing *really* works  (I had some descriptions that
>> were somewhat high-level; I wanted to obtain a more precise lower-level
>> understanding).
>> I wrote a document, just for myself, explaining what goes on in the code,
>> as I can’t even try to follow the Linux kernel without taking notes.
>>
>> After a first iteration, I shared it with the bufferbloat mailing list,
>> in the hope of getting corrections.
>> I did!  Most notably (but not only), Neal Cardwell helped me a ton - and
>> now the document should be quite thorough and hopefully correct.
>>
>> While I only did this for myself and just asked the list for help,
>> several people have in the meantime told me that this document is actually
>> valuable for the community - and so I thought I should share it here, too.
>> It lives as a Google doc at this URL:
>>
>> https://docs.google.com/document/d/1-uXnPDcVBKmg5krkG5wYBgaA2yLSFK_kZa7xGDWc7XU/edit?usp=sharing
>>
>> Comments or fixes are very welcome!
>> Please feel free to forward this as you want.
>>
>> I know that Google docs is not the format that we people here normally
>> use  :-)   well, if someone thinks that it would indeed be useful to write
>> this up as an I-D (probably skipping all the code details though), please
>> let me know - I can do that, and I can also present it if there’s interest
>> (but in Vancouver, not Brisbane).
>>
>> Cheers,
>> Michael
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm
>