Re: [L4s-discuss] dualpi2 & iproute2 - how to get latest L4S ECN coupled dual queue?

Neal Cardwell <ncardwell@google.com> Wed, 13 September 2023 14:47 UTC

Return-Path: <ncardwell@google.com>
X-Original-To: l4s-discuss@ietfa.amsl.com
Delivered-To: l4s-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B58A1C15198F for <l4s-discuss@ietfa.amsl.com>; Wed, 13 Sep 2023 07:47:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -17.606
X-Spam-Level:
X-Spam-Status: No, score=-17.606 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id enQMg_WQSfim for <l4s-discuss@ietfa.amsl.com>; Wed, 13 Sep 2023 07:47:36 -0700 (PDT)
Received: from mail-vk1-xa2d.google.com (mail-vk1-xa2d.google.com [IPv6:2607:f8b0:4864:20::a2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 91F26C15198B for <l4s-discuss@ietf.org>; Wed, 13 Sep 2023 07:47:36 -0700 (PDT)
Received: by mail-vk1-xa2d.google.com with SMTP id 71dfb90a1353d-493639d6173so2686943e0c.3 for <l4s-discuss@ietf.org>; Wed, 13 Sep 2023 07:47:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694616455; x=1695221255; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=V2XXic6hOAivKe7o8cntxA9pLjVK5u37OyHOOquCV48=; b=BILPPPC6n3th6+LmR8o5yqi7rHjhb97UBHlZskyenMS0DAmym9msrIAj1yZIcV+bX6 E8/BcMNW8wOBm9X1HnrQE6YyCWJm5jgz3siUAeGEnA2LfLE28NWj54AXvqaX2RxGHa4Y 7fuutRjfiWHIIHFWGU/sE2rwQBSiscAuTkNWancdFVVnkSoEX14UYqwf3EigG//DRWD2 LJwtXks2LUHm59AEGTgtyGkeU7fZHPgVohG6Qqz7HagcQGbXN5pigc+GiB1eMP7dG+x8 SiCAq8py0GLl9mVwniFTgTJM51Fm20eYkT1/kz2yAIT1API52nKxLQjQbMSeHDTTdZBN kRAw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694616455; x=1695221255; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=V2XXic6hOAivKe7o8cntxA9pLjVK5u37OyHOOquCV48=; b=Z2+l6kzdHzzd1DX4+gHwA9dZXRI5zBrYGXtoXShD/SngIvU1fd7QhL+u/JV8l4WhI2 dRNND+cPPvV9eUkNv3PLHecYRHRFnJBQgjm4eO1HSSBwqW2oUszAlFgBiYjz8jt8NFnz Lrv4a7LPwhyECQ5YXIWSQXfGv3u1zQy6G4edMsPM3988j7hS3Wq5RU3Y2nlSHee/A6Td zhrbavhfsiGycNJ/OpUa04ofnU3vkni1wMY5xQrf+MTaqcNz2gz/N3Anar4n/JvaWc6v ffG+djkXwhRQSCRQe7ivrAzsLA/wnGDV1VbKCyYn4wwV9Oux/1m5kPd1VbxQy3f2TK6o ByCw==
X-Gm-Message-State: AOJu0YzEQNX3nfdmLUwth6dw8PttMK43Qm9ry9wot5/xnZHa1yi7tgzE L4bfLExWY5JKsoTf/G6lkqbyvv3uTcdUW9U+hHVqwNMZhUVcWF+n4Qd6KDKf
X-Google-Smtp-Source: AGHT+IGyeCGqkRCZ0Af1ik9Ap17hBkMpeUv9M0yoNgGyssWtvTES99cs6WWIXjPDebNYM4yKgOwKQdMLlpA+1FZnpxM=
X-Received: by 2002:a1f:c504:0:b0:495:c362:94c1 with SMTP id v4-20020a1fc504000000b00495c36294c1mr2719220vkf.7.1694616455438; Wed, 13 Sep 2023 07:47:35 -0700 (PDT)
MIME-Version: 1.0
References: <f61972f2ccd69addef76002053a91237@rjmcmahon.com>
In-Reply-To: <f61972f2ccd69addef76002053a91237@rjmcmahon.com>
From: Neal Cardwell <ncardwell@google.com>
Date: Wed, 13 Sep 2023 10:47:18 -0400
Message-ID: <CADVnQym_VTzjPtvhcTOqgmH_FERijY06qCPc-eJXAPXgMbw4dQ@mail.gmail.com>
To: rjmcmahon <rjmcmahon=40rjmcmahon.com@dmarc.ietf.org>
Cc: l4s-discuss@ietf.org
Content-Type: multipart/alternative; boundary="000000000000dbd14b06053ea3d7"
Archived-At: <https://mailarchive.ietf.org/arch/msg/l4s-discuss/e4hTpNC_XowXLg4jLD4zUgoMxdg>
Subject: Re: [L4s-discuss] dualpi2 & iproute2 - how to get latest L4S ECN coupled dual queue?
X-BeenThere: l4s-discuss@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Low Latency, Low Loss, Scalable Throughput \(L4S\) " <l4s-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/l4s-discuss>, <mailto:l4s-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/l4s-discuss/>
List-Post: <mailto:l4s-discuss@ietf.org>
List-Help: <mailto:l4s-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/l4s-discuss>, <mailto:l4s-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Sep 2023 14:47:38 -0000

On Tue, Sep 12, 2023 at 4:46 PM rjmcmahon <rjmcmahon=
40rjmcmahon.com@dmarc.ietf.org> wrote:

> Hi All,
>
> I'm confused about how to best try dualpi2 on a modern kernel, e.g.
> Fedora 38, in a lab environment
>
> [rjmcmahon@fedora iproute2-6.5.0]$ uname -r
> 6.4.12-200.fc38.x86_64
>
> It looks that the code for tc qdisc dualpie2 (via ipoute2) is not
> released from kernel.org
>
> https://git.kernel.org/pub/scm/network/iproute2/iproute2.git
>
> The following web site looks inactive for years - is this the latest
> code and, if so, what's the debian released that it best matches?
>
> https://github.com/L4STeam/iproute2


Keep in mind that the https://github.com/L4STeam/iproute2 repository you
mention here is for the iproute2 package of user-space network
administration tools.

The dualpi2 puzzle has two main pieces:

(1) The kernel implementation of the dualpi2 qdisc, which is in the kernel
in the https://github.com/L4STeam/linux testing branch that Bob pointed you
at. You can install that kernel from the .deb packages pointed at in the
README in that page (recipe
<https://github.com/L4STeam/linux#installation-debian-derivatives>).

(2) The version of the  iproute2 package of user-space network
administration tools that knows about dualpi2, if you want to view and set
the parameters of the qdisc. That is the
https://github.com/L4STeam/iproute2 package
you are pointing at here. If you don't care about viewing and setting the
parameters of the qdisc (you just want to use the defaults and don't care
what those are, which is likely the case), then you don't need to download
and build the iproute2 package.

If you do want both the kernel and the iproute2 tools, you can use
something like the following recipe:

## install l4s testing kernel on test machine, using the .deb  packages
with the instructions here:
   https://github.com/L4STeam/linux#installation-debian-derivatives

## edit /etc/sysctl.conf and add a line at the end like the following
## to enable dualpi2 each time you boot:
net.core.default_qdisc=dualpi2

## reboot

## build iproute2 tool that knows about dualpi2 qdisc:
git clone https://github.com/L4STeam/iproute2.git
./configure
make
make install

## copy new tc binary to test host, if you built it on a machine other than
your test machine:
scp tc/tc $HOST:/tmp/

## on test host, view dualpi2 parameters with the new tc tool you built,
e.g.:
# /tmp/tc qdisc show
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev ens4 root
qdisc dualpi2 0: dev ens4 parent :2 limit 10000p target 15ms tupdate 16ms
alpha 0.156250 beta 3.195312 l4s_ect coupling_factor 2 drop_on_overload
step_thresh 1ms drop_dequeue split_gso classic_protection 10%
qdisc dualpi2 0: dev ens4 parent :1 limit 10000p target 15ms tupdate 16ms
alpha 0.156250 beta 3.195312 l4s_ect coupling_factor 2 drop_on_overload
step_thresh 1ms drop_dequeue split_gso classic_protection 10%

cheers,
neal

ps: if you don't build an updated iproute2 package, then when you use "tc"
(that has no knowledge of dualpi2) to view the qdiscs, you will get
something like the following:

# tc qdisc show
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev ens4 root
qdisc dualpi2 0: dev ens4 parent :2 [Unknown qdisc, optlen=104]
qdisc dualpi2 0: dev ens4 parent :1 [Unknown qdisc, optlen=104]


>
>
> Finally, I noticed Broadcom Jericho 3 merchant silicon DNX family has
> L4S ECN marking & couple dual queue support but I don't seem to be able
> to find a system integrator supporting it. (Maybe the generative AI is
> causing network engineers to ignore billions of end users and their
> devices that require low latency services?)
>
> Thanks,
> Bob
>
> --
> L4s-discuss mailing list
> L4s-discuss@ietf.org
> https://www.ietf.org/mailman/listinfo/l4s-discuss
>