Re: [tsvwg] qdisc hash customization as a path to shared RFC3168 queues

Greg White <g.white@CableLabs.com> Tue, 29 June 2021 20:32 UTC

Return-Path: <g.white@CableLabs.com>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4D22E3A0882 for <tsvwg@ietfa.amsl.com>; Tue, 29 Jun 2021 13:32:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.101
X-Spam-Level:
X-Spam-Status: No, score=-2.101 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=cablelabs.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ui4E4fliS0sk for <tsvwg@ietfa.amsl.com>; Tue, 29 Jun 2021 13:32:08 -0700 (PDT)
Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2110.outbound.protection.outlook.com [40.107.94.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1A6763A0884 for <tsvwg@ietf.org>; Tue, 29 Jun 2021 13:32:07 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d/nayItRojfpnYV5HUIDYfmliikwNZ2dDBlqU6A0C/yd6yZNIONjFJOMpxxtJGyRsSDjFQSgWkdUrZC+TBT1MkibDjThbOu2XobHEDsAAcfZvFbTpDiWqiME2f8lYt8SV5pcy5RVqTiAScGWgyKAiKz2Lnb6+jZITzq42YyGy5kNz86sT1Ad9gyuVpoKAFbMmvgXNc40nqgyUG4I1T6Kr9w278OVSaGL4+JlHnZjsmLW9gnpsNCO4zWsJ7RhsVnnFv+KYD/w+i5Ybeu6ITRtwgRjXX4kLoyMBoNorGpsGFHFpxVkCcAWaxe5vG0qfTaM+wWHRshu16ISrLDGSNxSng==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v2c9/bhHphof0Z0xPx93BTlmG31+JKdFGd4buFrTywE=; b=Bq7U1TY9/C2Cw+ZpKHm4VM+EI3qAUYh6IDRVRMGEcWBEVXdUcrzBly1s2HTYmMOoTUBPQlwLrcxZV4Ibw/GRYYw9dcPXNdPmLKH2vJZ2bX6ZyI7E3kjHxPYdlTkywLjgnJyrg9RHiyZaZyqLxxWMK23kopZ0dV1dQcGFpvD3lrEkkXKOZ0C/t5X8SZ2HZkLQ3CmZcNDZpZgs6XvPX0X7YihAfY+OIOFAJm29KD6E1mugnlpKqBROY5CgLhJFv8HPTxMEnjf79sHWuiz4nunwRHWQQktNP3TJXmHaIIgX8PAxioBBXQmAZxWXFW9Ah0k+C9ucftV6SebnT/ZPXjKu4Q==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cablelabs.com; dmarc=pass action=none header.from=cablelabs.com; dkim=pass header.d=cablelabs.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cablelabs.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v2c9/bhHphof0Z0xPx93BTlmG31+JKdFGd4buFrTywE=; b=kZmJ8JXOk8JaMexgBZUoaoKuX40mPop4ikk9VnW8GMjLtBeXhx850MTNUyPMkyjVk2ObhGNa8xwl0g25TJFMc0xZ2nnqxBDzV1QuuDkG/t2a0RPVIaGzMddYqiHBhFNF9P1G1NkeroeeyH1SKiLd+ooaoKWFaFGZFyJvqMSkisd1KZNaC4E1CqDPYQGm8jBDLTTx40kJn9mOQsV/dBJbvbhDbKftClY/6RQZK+DcozBCw59GqTWZCVmGAc8YExbNGaksqcGqKDGAWFkLcRu1TfMmm4AqBNiFxAY92etoa5JnYSbaCKJqiBhACCnT3HeC2Ae7AfSUX1rfQkrNqMK3rA==
Received: from SJ0PR06MB7861.namprd06.prod.outlook.com (2603:10b6:a03:38c::19) by BY3PR06MB8066.namprd06.prod.outlook.com (2603:10b6:a03:3c5::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.21; Tue, 29 Jun 2021 20:32:04 +0000
Received: from SJ0PR06MB7861.namprd06.prod.outlook.com ([fe80::c138:2adb:9183:1bd]) by SJ0PR06MB7861.namprd06.prod.outlook.com ([fe80::c138:2adb:9183:1bd%5]) with mapi id 15.20.4242.023; Tue, 29 Jun 2021 20:32:04 +0000
From: Greg White <g.white@CableLabs.com>
To: Pete Heist <pete@heistp.net>, tsvwg IETF list <tsvwg@ietf.org>
Thread-Topic: [tsvwg] qdisc hash customization as a path to shared RFC3168 queues
Thread-Index: AQHXbMrv/Jx4Hg9mSkWu0tj2dkNMVqsrDXiA
Date: Tue, 29 Jun 2021 20:32:04 +0000
Message-ID: <F356C8A8-33D9-4BD9-8B93-4AE6A1840C58@cablelabs.com>
References: <89c90eb3000ea6cb0461735c16a57c016cc4e6f9.camel@heistp.net>
In-Reply-To: <89c90eb3000ea6cb0461735c16a57c016cc4e6f9.camel@heistp.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/16.44.20121301
authentication-results: heistp.net; dkim=none (message not signed) header.d=none;heistp.net; dmarc=none action=none header.from=CableLabs.com;
x-originating-ip: [73.243.9.160]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 53804354-a7d1-4011-c923-08d93b3cf505
x-ms-traffictypediagnostic: BY3PR06MB8066:
x-microsoft-antispam-prvs: <BY3PR06MB8066BF5A74D7FDF4EB4ABD10EE029@BY3PR06MB8066.namprd06.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 7dsVxQg7Hv9Uyk0x8d+b7vOkC4bK+NrZlhFjskOEtcPp8l/7SdJDyyjQx3tBHR6ahoYP9hL3Y1iEkTonTCfjOt2cThurnYkKuudHTTkxk/vnRatHQp8vdxxPt4/mnJGNdlHrkv9V4EBOn7Y41ClBodatmLLoA6NFhdlmU/oahzBvh7nPEZHaEyZU656dqKbPlcu4CAuoyhKRzaHQ7NJNNPeQH47nJZ/s/tb9G0XY9EnbB9LldiZRtdkhb6dbSNxGjmtp+EMLj68v1i6EUo2Tdq4SKQqfMOtULaJVdpBerze1kXTyf89YVF5m5uJzrfEYWJcUeMNpbdcrsbMGQ9GpB76naHxhtIS4QRBEaP2VeLKojGwrvP6Go1nRjNyN/ugiJ0p1iVAoYDF1c3vQ05yEp4CsljtQY9YYiS1WYANjGu2b538ZqeBdo/vfxQZb70goBndbwcadjwbApIc788NEhk+GZPqIeKR1W42hwzFknmgB4UxX1RkqIZpV1fA4ObaTH5eI8VBn44a7uEbBuHVSAMPH5yy9u4oel5kCbYvFXlKqxCZkxH5IR6UgWzTGL0EYos+onoHSHYymHU/dknRLPZ16DopMSAY8mDddC7WYrsMrMZ4zL0mmRVuNNfPrggIl+TEoqj/9Ixfdo5vATPSeZS/hPBONOVLqjg+2FJVmbSSgkHXKPoVkv9Q5yhfbPMb8gKFlUzs4u+hElH3EKfPm/s7OJsViTQdQ/qxbgWllfbl0e6lHSce6Y0hqqx9/O08qJCQpgE+kTIpKAU9I5ViRmbnoD7vxNoH9dAqeqfja2boczMRD4W4uGsjCHF3YWtGFjDGxU/qEuKNEIIdi2ssQqER9pkuUV8E/HoiekgJvenM=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR06MB7861.namprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(396003)(39850400004)(376002)(366004)(346002)(136003)(122000001)(26005)(8676002)(71200400001)(38100700002)(6506007)(5660300002)(110136005)(53546011)(36756003)(8936002)(33656002)(316002)(66556008)(64756008)(2906002)(966005)(6512007)(478600001)(76116006)(6486002)(66946007)(186003)(86362001)(2616005)(66476007)(91956017)(66446008)(45980500001)(85282002); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: mZSYVJWUaVd/dWKSj2P+QOhXeIwnnG2pjC10Qg1MzkM/ILRGNd0Eicicr/06EvV7UGGh4+WTd1TzD2MEcL07qzeWtqHh/eIslrI02VbynZE9H2g8UOzBDlW2rx8OUUmhPl/mZvz/bmSaGTevVF64NM81tPxn3NqpbqIYLCYK9qKh7XIpOW9LhBmX3hyryC348LHQ+Mpg7kvUFWNVO5f6rx3m+7MUJXmdsYyZ+AfXVbuP+nTmBf8XdbL94vN41MJAODoJgsazKvGs84yHnn4J0y/2GUoccmxjOnyrGU7nnrmshiX6eKsrH+lmwtjnUAWVh4iyofSm1yMAITdUIJEl6s1aiAJeCAknyRZK8OTy99pENMkxGpQ/1TcD9YuhttukbQ+fk4NesIo8xgL7+IWjDfEPEbpunmH+JmKgY5jsGcbVeCymF7vDOASNh9LXp3KukEiGBqaPeUoY9oHFCBWq7iO4I/ugDiAMv6vG/SZlEtQlzQ5PD3qEUso4vS/j1SdCwizI+CacFCi4G6zL1P4hSUdgvnBKnp40gyCoxHtZl3QnuJWV8U4UaQNDZy7QFrDeTQyj6FubNs9f8TXV5tCKdFz41VddpQpXKgKprVirolBr7wAHM9p1PLjNYG9MzN36WejMADt//FObsacaC5eWQFLvHBzMsTrCN6ktjqq+E3Nw/SUdGAYx59aArsxt/G80pBkIf8jnOhtDOJQEHZtAcj27b30UfVJrkSbqgrN826olsbwgT5VxL7/2ESAv7tj16gqgbLLHtkrTSniZgdpBfv8MSasDYXXi5B0NvmRWBQWJYYqiBmihwE2agooDrrWRzgzimVxSX++3kQZGylTF5/MuZyBbO9WsUHaaTFnX8MHxfqFI7MXrp8uLUGMltDiGxDtv/VuIgMCgV1efn97h2APQcfE+C8/kUMfWRp61hF34aPxf6bhTX+4J9drffo0v7HCmS09K3jzZF3ZeZ594qyoeSY2Dm0PpI2WjU2+fPSQPNdWsc2U3+1rR0/Np3Qeqbe1Y12h/gYmJR+Wrc6NffU7VnuR3OShGNC/kcJzR3gc0uSnGSxShhsGYu9OMQbSVKWNY1tnL5uhIWEbXdjAe++K5JR0G1She10hk/jtXmahO9ma+ClW6F2u7AjBQIwZNc5BYATJLQtrBcBQSVHlhbaS3O1F4a2cLa4cka+9VewEkFZP1aJRCfGuueqfeKhulbdcQq9twKEZoUx4FllBCsFdm1BkCH0D6qk87BOFdSEg8s2zjVonRG0AjC7xuDuy1Q2oEV+/WzIloxAEyjEgWSe7oG69tZhAhUsREWVl7ayE5oxKMhqsI2Qlal1RKfThA
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-ID: <83480868865E954590CA982636F22A53@namprd06.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: cablelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR06MB7861.namprd06.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 53804354-a7d1-4011-c923-08d93b3cf505
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Jun 2021 20:32:04.2040 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: ce4fbcd1-1d81-4af0-ad0b-2998c441e160
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vzkRoAYx2j86uEIOe7wnBR139cfMSteVRwMjLYg6AL/bsQNAiODYNkUwKM2+tSEB31N3lHrnUjWcCr2rS3V9MQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY3PR06MB8066
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/dzaED3reev6lVKdm7QLmcrY9RFg>
Subject: Re: [tsvwg] qdisc hash customization as a path to shared RFC3168 queues
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 29 Jun 2021 20:32:13 -0000

On 6/29/21, 3:41 AM, "tsvwg on behalf of Pete Heist" <tsvwg-bounces@ietf.org on behalf of pete@heistp.net> wrote:

    In the L4S safety discussion we've talked about ways that L4S and non-
    L4S traffic can wind up in the same RFC3168 queue, even with FQ:

    https://github.com/heistp/l4s-tests/#unsafety-in-shared-rfc3168-queues

    I don't think we've mentioned that the hash used for Linux's FQ qdiscs
    (fq_codel and cake included) can be customized, for example to provide
    host fairness, "subscriber" fairness, or otherwise. I wrote this up
    with a test script showing some ways to use it:

    https://github.com/heistp/qdisc-custom-classification/

    It's easy to see how this could lead to L4S and non-L4S traffic sharing
    the same queue. If an operator does not want flow-fairness, but wants
    fairness e.g. among destination hosts, this can be done by attaching a
    flow filter to fq_codel (where 10: is the fq_codel handle):

    tc filter add dev enp1s0 protocol all parent 10: handle 1 \
            flow hash keys dst divisor 1024

    That would cause all traffic to a given IP to share one queue.

    This has been in the wild for some years, so it might be worth adding
    to the l4sops draft.

    Pete



Hi Pete,

Yes, I can add a comment on this in Section 6 of L4Sops.  The current text in that section equates FQ with 'flow-queuing' (as opposed to host-queuing or *-queuing). I think I can fairly easily add a note about non default hashing. 

What do you think about this, at the end of section 6:

<t>Note that some FQ schedulers can be configured to intentionally aggregate multiple flows into each queue. This might be used, for instance, to implement per-user or per-host fairness rather than per-flow fairness.  In this case, if the flow aggregates contain a mix of Classic and L4S traffic, one would expect to see the same potential unfairness as is seen in the FIFO case. The same remedies mentioned above would apply in this case as well.</t>


-Greg