Re: [tsvwg] ECN encapsulation draft - proposed resolution

"De Schepper, Koen (Nokia - BE/Antwerp)" <koen.de_schepper@nokia-bell-labs.com> Fri, 11 June 2021 09:00 UTC

Return-Path: <koen.de_schepper@nokia-bell-labs.com>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED82F3A2F9B for <tsvwg@ietfa.amsl.com>; Fri, 11 Jun 2021 02:00:05 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.698, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FKZQRZyLFf9U for <tsvwg@ietfa.amsl.com>; Fri, 11 Jun 2021 02:00:00 -0700 (PDT)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2104.outbound.protection.outlook.com [40.107.20.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 70E233A2F99 for <tsvwg@ietf.org>; Fri, 11 Jun 2021 02:00:00 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Mc6h6Vkli8tlwsGOHbxmGLKWFsp+o15pjpEUO3f26suJrIgjB/ojXcZfauWd1hACen/JxyYjI+xDAM95zfwhC3DdWJFdkowcaJDFTt2HsvknEzw0vPehwPU5woGWkrYHhLuvLOyQYntcEtnkbCWJJqULmxjUW3H4gBbnEOqLYPqeBxs/emLy0MrnfR/rjObICqpQ/sggXrjv8jLhAp0UughTCckuiDz2kCy8F2pUdkDjCYjKDMID6Rhdr30TBj6MSzlHUTvfISfWwz4fHiFwCUz2Eipld9350jZvCBO0i3d3769GfTP5SoxAk1n7PXPmwhYbKyBIy57YBYtPpcjmdw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GJyFCUKRCTXxP7eCyUOAbgJuDqFZoUALynY6DcxGna8=; b=DPeuPY42J09rhueLQA5TYUS2vCW5YLbyZHfud0yd1FdFMTFPBuXh4mtm249LHXUCzYoO9LV36E6wQSjQJ7CAeBpv8gML7vmYdi4MjHRsKY20yuA4Jbv2UDSbJjR8IVrDEuwm0t3tWNJfG0akOmxRgEFKsDyBxFQo4N4Z1Zq19XN6PUYY456apOVCJgZvJOSx+FIG6vbqz9ovO1WfTJ+/J7r3QmbRFSvZuTt1Uw9AM8aBDXyvi9OCKHAJp1m5Qs/QKLrInk9MiprtCx6m5Y3DRZDzP1nkAt5PrEWylHzf0oMQ+pP6CUnqokHjYlyhYkjUlxEeI4/GMQSaNHSIL2r2tw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia-bell-labs.com; dmarc=pass action=none header.from=nokia-bell-labs.com; dkim=pass header.d=nokia-bell-labs.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GJyFCUKRCTXxP7eCyUOAbgJuDqFZoUALynY6DcxGna8=; b=KX9ITl3E5fJBp4W61C1LW5DtevQaj4bXioo0OI14BwFx/haY4fM+oFXOtPdnaPrYQ3mpz10Rp2HOhtfizDFwIikZBEb1UQ2dtREAb7s0H886szFc0LJfLND1Er498JogGTCP6Gupdng05dubUDZ9iLal8wS6ZSohVLXw0QdvU4Q=
Received: from AM9PR07MB7313.eurprd07.prod.outlook.com (2603:10a6:20b:2c6::19) by AM4PR07MB3316.eurprd07.prod.outlook.com (2603:10a6:205:5::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.10; Fri, 11 Jun 2021 08:59:57 +0000
Received: from AM9PR07MB7313.eurprd07.prod.outlook.com ([fe80::d85a:c159:4d35:21b2]) by AM9PR07MB7313.eurprd07.prod.outlook.com ([fe80::d85a:c159:4d35:21b2%8]) with mapi id 15.20.4242.012; Fri, 11 Jun 2021 08:59:57 +0000
From: "De Schepper, Koen (Nokia - BE/Antwerp)" <koen.de_schepper@nokia-bell-labs.com>
To: Jonathan Morton <chromatix99@gmail.com>
CC: Bob Briscoe <ietf@bobbriscoe.net>, "tsvwg@ietf.org" <tsvwg@ietf.org>, Markku Kojo <kojo@cs.helsinki.fi>
Thread-Topic: [tsvwg] ECN encapsulation draft - proposed resolution
Thread-Index: AddOnb5B4hPq8TIqQMeaLOUn3OP2OQB/ZnoAADX2kQAAAZDhsAG8/UEAAHx9tgAACEA9gACv6mXAAAmyboAAOj9EoAANYkAAAAN8j2A=
Date: Fri, 11 Jun 2021 08:59:57 +0000
Message-ID: <AM9PR07MB7313DC2C4F922A90E2E602BFB9349@AM9PR07MB7313.eurprd07.prod.outlook.com>
References: <MN2PR19MB40454BC50161943BC33AAAD783289@MN2PR19MB4045.namprd19.prod.outlook.com> <43e89761-d168-1eca-20ce-86aa574bd17a@bobbriscoe.net> <de8d355d-08b6-34fb-a6cc-56755c9a11ee@bobbriscoe.net> <MN2PR19MB4045DB9D2C45066AEB0762DB83259@MN2PR19MB4045.namprd19.prod.outlook.com> <alpine.DEB.2.21.2106021717300.4214@hp8x-60.cs.helsinki.fi> <290e1624-fa1e-21d7-95fb-90e284c27dd8@bobbriscoe.net> <C7509065-526C-4712-B6CD-E919910E280E@gmail.com> <AM9PR07MB7313E7797F850B210EC3A799B9369@AM9PR07MB7313.eurprd07.prod.outlook.com> <3009F41B-1D79-4B2D-BC16-8F2049EA4976@gmail.com> <AM9PR07MB731372180EBF5C815A684F3CB9359@AM9PR07MB7313.eurprd07.prod.outlook.com> <51F33C7C-3323-4ADA-98BB-16A83F763FCE@gmail.com>
In-Reply-To: <51F33C7C-3323-4ADA-98BB-16A83F763FCE@gmail.com>
Accept-Language: nl-BE, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=nokia-bell-labs.com;
x-originating-ip: [2a02:1810:1e00:cb00:6c54:726e:5266:66c0]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ef586e29-879c-42b4-7aa5-08d92cb749d6
x-ms-traffictypediagnostic: AM4PR07MB3316:
x-microsoft-antispam-prvs: <AM4PR07MB3316BE159966E55DF64289B3B9349@AM4PR07MB3316.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8273;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: App6G+DqSYJmEXgPZDhsFObie+FHwrFfYuatbyDLxPwM8k6/AtxqS+TacANE+iTobWn3ySaw1JzW0GrFkrFynF4GmlAs96TdkShkCnj1poMqrCHwaNH9h1+JtrOrhUxgWa89bzx9hTp6ZXUqeU1OGKUFm6nglSjFOB2DCacX+OHrbGVMsbeYTP2L5gxCfxZaKHmi30csaitBX3236qw8dLfwIESHfL+pqRk3qrP2b0UD7Hnw/xBX1rHSDbBfQ444oL69m27oGRWb5qzQp4RPtEKSDOGFb49vdU13177NOiLwlH8hqJTzn6GYMoyI5bxSsk52SfOIxkDdk6f+pW4vWBPICR/V24hD9PtFsduDYKD9/rOROdOKc20H2bwjtH7MI/0y9d1yTQGJpfQY8YITF++/2OBOYD+0E8IY55bpFYZC7GEbO4eQYTLWiVu7SA+NasusuW1YAhrkZSqm6ftdDTS/FHUVGSdbTomntC6OWMCxbj7imjRL3KdvG+iYAWRPryzI6gXm20KaL84tbz59F8rtMrBZWmBzkMGCboCQ72D7DnaiJgJUC0qMx3xFKQkPtltccKQTNncMmXcJkCttiHpOFrx8axw01/jKXR9wupw=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM9PR07MB7313.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(39860400002)(346002)(366004)(136003)(76116006)(66574015)(83380400001)(8936002)(316002)(8676002)(4326008)(71200400001)(9686003)(55016002)(86362001)(54906003)(64756008)(66556008)(66946007)(66476007)(66446008)(478600001)(2906002)(33656002)(52536014)(6916009)(5660300002)(6506007)(38100700002)(53546011)(122000001)(186003)(7696005); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: POWMMSxHv1wUb4U0hgptHtF9PtgMRllFW5eC3BVvAeaFaDiXr165mjZNgwmXyic4JTBGzH6ikQx7qP8DRCMZpxRfP2djzfE5L5kQjjHAoAGZH6cUH61DRxDaUls3S6702RDYQzyWADu5I1O3rWsMcWU/DSASjcmKgZYBKoB1q2ObzHugDRct1TLLhGe/5P+b7pDm50i0MJmdEDwSXQtiRIKzgXTW2qHHLfNZcV1+JPI8RFbYiCFhnRb5TCuIH7pjTbVUhqqTsAVPgTOtNU0IQ9EfbRwgythWYKhhyvV9M8wHht3BRAaSxQboM5dqVTY4VzpjTUCyS33aYpfYEMUJR4OK5vgh58+Z9s0RgAvpxfSoCFJ6Vo7NzNgoow6DtUt+KASjXCx3gw8LADLfOsrqPS7+0f4lQKpfVu21TQb8HU6I2dM3aXdfM7Y+x+zHzGX+T2aXvU82Pt3OJUccuT0sOr1p4TlaWpsxFTb1DI1EDvXMqFG11ro8OhnLD2+e5bMYFemYUGRYWopSnL9i1bqzJVComTAjXHlzs5f5F+Rf971RDJCCC1EZTs2hZsjqb4nmGCaJOhf+PRHjSq+I7wBkhzLUSdzwOwI/hBP2GgbmbNeCC4santavdyzdOjFZX3ggmvg7pM2+97k2LxpfotLEzn+jfIT+9qwU32zFQhphiCVpFlRYsxx3h6LA/QVj20NTA82kmN/WkRfVvQUdQKUWsjVjVbwhcyRH3BLS4xfImi2rwxTwjJNyrwMvddp+1w3XUk9RTe7h/cWGCQCA1K7S4KLhz5lzElgBQ485qRqrXer5kl+4jFBbft6KzfEeeAkQRfpZf0CdIHmrEgiESvrDIKnS94AalJOieVzGU5X621xWjaHFRO6TxOEpvh4RF0D/7qtp2MMS2vroK8B7O3VEYC3lloghv51d9svAj7zqBCHHxt5EYzVimAoFX40D3ER2+xJmRzYByiTrlrwiL2JukWtnKQsuucNqeg0n4j6MuL6jAbyjQQjajV0gY1ZsdzNFkBp2zgnuq072tNVvysxsyIjwXZmib8XqF8sLsC8/MYFcNIpE2rOQmxM+wC4FkAq2Z9Evb1mZTKdRbu8O6fzObFKlWVtotSmdOE69dM8x3vvNoTLGBqorIuh+1H1sbD3E1lmTHsO7UAXFc9wSvOPF4ASzJYvIh5qc3De8u8LlyswCq09iqZCasFyeIxT6/TF7BsbZiiO9PW9RvyKtoXQYnUm4NRuj6zHqYc2JQQ52EIgcUB7jWZSzCOwtltwutnNRcqH8RCFbZeG19CRde5wT9J2291Sy893N5p0SK2xApzZCRh5oRjs5uiOZxVBbBiz7/lkIvWVwzi7q+RBJcrQFMDFrL2S3xhzwooZRSuSKpxm7K7PmC7bwGtbC6GyoRZlL
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: nokia-bell-labs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM9PR07MB7313.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ef586e29-879c-42b4-7aa5-08d92cb749d6
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Jun 2021 08:59:57.8010 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: y5+U9bUqo9T1gP5I0973UeJ/LX68ocYfh8FCPMM+Wqcr9FAbyErOGM5RunpZuNAbdYgrXqkTo7kkH0estN8AqMRuVxaoDE8ARnfUYwXrDByO8qWJ2Jnw5aooCbiMvpJR
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR07MB3316
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/zeYN6pcta2_ALPXSAJLZRAdDvrM>
Subject: Re: [tsvwg] ECN encapsulation draft - proposed resolution
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 11 Jun 2021 09:00:06 -0000

Did you consider the consequences too? With your approach it means that if you just mark 3% of the ATM cells, 100% of the inner 1500 bytes packets will get marked?
Is that what you intended?
While if there are small packets mixed in the inner stream of say 48 bytes size, they still get 3% marked. Is this what you intended too?

It all depends on how these marks are used by the congestion controller and what the Network congestion is related to: a bit-rate limit or a packet-rate limit.
If the outer packet size is constant (always a full MSS or ATM cell size) it doesn't matter if the network limit is packet- or bit-rate. Then it depends on the congestion control in the end-system. First does it keep a packet based window, or a byte based window (I assume byte based is usually used now). And then are CE's counted in "at least one mark" per RTT as in Reno/Cubic or in % of bytes marked to the total received (as in DCTCP-like).

As long as all combinations are possible, there won't be a good answer. We should decide what is the simplest for the expected way forward (scalable, marked bytes %). As we are going to have multiple marks per RTT or time interval, and those are measured in the ratio (bytes marked)/(bytes received or even bytes non-marked) I propose to keep that as the future recommendation, and decapsulation should keep this constant. In the near overload situation, which your example can be classified if packets get marked 100%, proper AQMs need to switch to drop anyway, which is rather matching your proposal I think.

Koen.

-----Original Message-----
From: Jonathan Morton <chromatix99@gmail.com> 
Sent: Friday, June 11, 2021 7:48 AM
To: De Schepper, Koen (Nokia - BE/Antwerp) <koen.de_schepper@nokia-bell-labs.com>
Cc: Bob Briscoe <ietf@bobbriscoe.net>; tsvwg@ietf.org; Markku Kojo <kojo@cs.helsinki.fi>
Subject: Re: [tsvwg] ECN encapsulation draft - proposed resolution

>>> So for me following simple algorithms would be ok to use:
>>> - only mark if the first (even partial) segment of the reassembled packet is marked (ignore marks on any other segments)
>> 
>> This would mean that any mark applied to a segment that covered only the middle or tail of a packet would reliably have no effect.  I firmly disagree with that approach, due to the possibility of accidentally triggering pathological conditions where a long series of marks is lost.
> 
> It could as well mean that one outer packet marks 2 inner packets. Note that AQMs best apply random marking and that any pathological conditions will be quickly broken by the closed loop.

I would class that as a poor workaround, since you are effectively excluding an entire, successful class of AQM algorithms from use with this approach.

>>> - apply a random probability on the packet based on the ratio of marked bytes in the packet and full packet size
>> 
>> I can see where you're going with that, but I maintain - based on my analysis a couple of posts ago - that maintaining the number of marked bytes is simply the wrong approach.
>> 
>> The interval *between* marks, measured in either bytes or time, is the important property to preserve; this is actually easier to achieve and is natural to perform statelessly.  If the AQM device can rely on that behaviour, it can be designed to perform byte-mode or time-domain marking to match, which dovetails nicely with all existing transport implementations (whether with high-fidelity or conventional congestion control).
> 
> These algorithm preserve the marking probability (whether bytes or packets), which boils down to exactly preserving the interval between marks. Not sure what the difference is?? The interval i=1/p with p the probability. So preserving the probability, preserves the interval... not?

You are (approximately) preserving the number of marked bytes and the interval *in packets* between marks.  That is *not at all* the same thing as preserving the interval *in bytes or in time* between marks, given that the size of each packet changes at reassembly.

To ram this point home - since that is clearly required for understanding - let's take an extreme but realistic example: encapsulating 1500-byte IP packets into 53-byte ATM cells using PPPoA encapsulation (10-byte overhead per IP packet), and transmitting them over a CBR 12Mbps ADSL link (28,300 cells per second).  We will also assume there's a few smaller IP packets mixed in from application-limited flows.  Each ATM cell takes a 48-byte portion of the PPPoA-encapsulated payload, so it requires 32 of them to encapsulate one full-size PPPoA frame, and we can transmit 884 of those per second.  We will further assume, perhaps less realistically, that we have a way to apply congestion signals to individual ATM cells.

In your world, probabilities are sacrosanct.  So at some given AQM state, we apply a 1% marking rate to the ATM cells, and you expect that to be converted into 1% of IP packets being marked.  But that is also converting 283 marked ATM cells per second into just 8.84 marked IP packets per second, and increasing the byte interval between marks from 5300 to 150000.  You can see here that packet intervals are *not* the same as byte or time intervals.

In the world of Reno and CUBIC and Codel that actually makes up much of the Internet today, and even in the world of DCTCP, it is the byte/time interval between marks that is sacrosanct.  Neither of your algorithms above *even approximately* preserves this property in the scenario given (which doesn't even involve frames spanning packets, but the arithmetic is similar).  The RFC-3168 rule, however, *does* preserve it.

 - Jonathan Morton