Re: [Detnet] Packet Ordering Function (draft-varga-detnet-pof-00.txt)

Balázs Varga A <balazs.a.varga@ericsson.com> Thu, 22 April 2021 08:10 UTC

Return-Path: <balazs.a.varga@ericsson.com>
X-Original-To: detnet@ietfa.amsl.com
Delivered-To: detnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 64E0A3A1C6F for <detnet@ietfa.amsl.com>; Thu, 22 Apr 2021 01:10:31 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, WEIRD_QUOTING=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8l6p4k0Snjvl for <detnet@ietfa.amsl.com>; Thu, 22 Apr 2021 01:10:26 -0700 (PDT)
Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-eopbgr80071.outbound.protection.outlook.com [40.107.8.71]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 284AF3A1C6B for <detnet@ietf.org>; Thu, 22 Apr 2021 01:10:26 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B9HxxNzGWyhGh5Rko0Y7qrpecViJ75YM2EgO2MV8KP+VCn3f/xi+vAIdIF0HD9JOOjtbx5FyVLf5L4nc8DzvYq5vJVYwnG9ydZ22vTFUV+Okp3dlAjJzq/FuQNsekGi0s+PWLpeDtOumuem1AgxTl42EtoqjVRNGlpmYIAJAHCjLb7gr7OZelx8mQBWK4yok6p73/g6x6oKG+yzT0POXtRqMf3BSxNEyl840ViTRDE61eRjbbOzqKZF6alvzhoWXE2vEaJ6/MsofUuG6yqEp3neP8FCUgKWqPrdQQ7051GwsPEdvfKf7FKclctaKiYxDllLNYf2Tr2zV+d9VbObU1A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V8Q72cSVsSZRD4tUrSwR+7jptU9onrofnw5d9APXT18=; b=BBhrDfJ3R07W06mj9To2QRO4Nb/lBGyeUK7YX/xBYB772iHYPz3QF5gDMZasxx2M1j9iCVVzyPJOxEVAjj+VqAtu8Fn9GLBsEPsgGK8A0OWPvtJKVHfhBVuKBJaMgnCPGSsVznPaWnGHolmltME7mS96H5v5p0eShJ5VzdThypnnIPa58slYGAszrdxJSzlNRl6z7p1+qz8LQxS7grULhIBE5l3aZ5e7V3gsQ/QSQxz12D1P5CmVXfjifJppKzFe+j1x1m1NFn+382vgZ24OpUYLZd+PAfe8GNb9hcjPEMnkv3yXD1ALrTe/cKSV1ofnN2oNz4fCGlIypYs2mjKVyQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=V8Q72cSVsSZRD4tUrSwR+7jptU9onrofnw5d9APXT18=; b=SIQVNpONWDhzYMoYy5EvW7mBm8SCx/rZdg1KjfNTwknl/S4EsFrk1jRyplEbUNNzEPJ2p7RNGvT6NxOunN9aUSUnQ7pHafbbI94Y8+v8BHXGZ5NHUNFRA6xgrF8FSL3k7XBy1JDDghFUxWWBsS4qHFaJWiXnZ0f3YXmN0OKeszw=
Received: from AM0PR0702MB3603.eurprd07.prod.outlook.com (2603:10a6:208:22::25) by AM9PR07MB7843.eurprd07.prod.outlook.com (2603:10a6:20b:2fc::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.8; Thu, 22 Apr 2021 08:10:22 +0000
Received: from AM0PR0702MB3603.eurprd07.prod.outlook.com ([fe80::c110:dde2:b70e:6e3f]) by AM0PR0702MB3603.eurprd07.prod.outlook.com ([fe80::c110:dde2:b70e:6e3f%3]) with mapi id 15.20.4020.017; Thu, 22 Apr 2021 08:10:22 +0000
From: Balázs Varga A <balazs.a.varga@ericsson.com>
To: Ludovic Thomas <ludovic.thomas@woolab.fr>, "detnet@ietf.org" <detnet@ietf.org>
Thread-Topic: [Detnet] Packet Ordering Function (draft-varga-detnet-pof-00.txt)
Thread-Index: AQHXNou9ZP3S1c809kGRQIDb43RSOarAFC/w
Date: Thu, 22 Apr 2021 08:10:22 +0000
Message-ID: <AM0PR0702MB36035772105916DF3C993792AC469@AM0PR0702MB3603.eurprd07.prod.outlook.com>
References: <2af69bf1-d5a1-c5c3-9a7c-9efd0bcecf2c@woolab.fr>
In-Reply-To: <2af69bf1-d5a1-c5c3-9a7c-9efd0bcecf2c@woolab.fr>
Accept-Language: hu-HU, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: woolab.fr; dkim=none (message not signed) header.d=none;woolab.fr; dmarc=none action=none header.from=ericsson.com;
x-originating-ip: [217.197.176.6]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: ec426d16-5079-428c-de51-08d9056613c9
x-ms-traffictypediagnostic: AM9PR07MB7843:
x-microsoft-antispam-prvs: <AM9PR07MB7843DC60BDF8F72BD9CE3AC9AC469@AM9PR07MB7843.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: /crQvWhiOyfIVI5CRDtX2QdakgFxGBRNJOi4SmYsZaFxvdHZgvJed6ryit6idP7B4y1f6p0cCKGWfXSwq50T3UTZtY5raACPsF52Mz6a6XbfMP4fE55el03TqvzVOrk9TdSIAgiYukT58NH8eN9dP69xCn9XjmEcQC59mNlD2Jp4Y7a2UWiRFpS7Nvao0YX21OeLoseYQnvMD155YSK2nqDhp+kIbpSYd8G0SNM7OOtvW657FCFHbUBjB3SSxB3nahtlCpVrn/ZHy7ht5X+dimvyiJ/59+EMMyHGIU9xvtrVSuC8V3tN2YyAgFPzP2/bO2a8T2HgE6QDnlxLIn8SA8KZLHdG9EMFyoiRpYj8jZlbqVZ+yoR+tqF9/EFSQ1KvY3YEbp5gQjGhDbuqI8laPQia3xQluAvjz0L0vZF/4ErXpMMXoTLobDrx/1c1G96O/6SrPkoQ5S2C9R3x2JujSwCpAGb/oFuRjMx9/8O59XIA/FraL+ZlhqfZ5YgbIMymgxgBULEAvQyf18DuS+PYCv2PBY2snMyTTg17BfQ/OO4GrVudApSgVY00dL1CkL+IJ6msC1mm/BaLJC+aq9KH40hdbHJ66j1DVucP3qQ+M5BjCjRZTm+QhjBRpApXKEiSVG6cNLodKMPXMZl1XBxk6sf/qc4liJPt7zOnS93Qtkk/Fgh5KMD/itPt4RYBNE0e
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR0702MB3603.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(376002)(346002)(396003)(366004)(316002)(7696005)(6506007)(53546011)(8936002)(9686003)(55016002)(478600001)(38100700002)(83380400001)(2906002)(86362001)(26005)(122000001)(186003)(5660300002)(85182001)(64756008)(76116006)(9326002)(66446008)(66476007)(966005)(8676002)(66556008)(52536014)(66946007)(33656002)(110136005)(166002)(71200400001)(85202003); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: 8RZyWb+LBa8qPgfygaf+9pRV4+U9oTCWCx4jAzfjcMQ2qeVKDeRrZTiRHGUBu2r68cgxoey78Dw1uaXegdO7USMorNOBQ0ZGc25CnkbnurswbcRZsB7Au+5Y1p+5r6pV5QS0Y1WICz6KaEhIPwhCZC4k1H1xE15FdnBvw1DOVpD0/Fv6XmxQqsTLXhrkA0mLR5pzQJoxonCLNGldLLLc0rDmyeVIfaGEnURGi5tr+oHcQRieOptez09+8eAqC4SyFMRWuUrTnzECGB7BJUuaTWTzKsMSdXSyXwB8PSxY8+23n4s8Azq8XRYd3J6oFlwEyT96ZFeiiwuOz65ykKjStvzU2ziK1GmBu4D5pyVlPYwGO4DT8eejQ2388YZSNopr/v9bzvWFmPw+wo9b9Y9c4bz4mXbKQ21Y3UgAoQBumq+0/IAjmTpTMU3/grczzZxG8E51J0KfPvHS6VERLG8PVV62NgMXJCVfqaQP7OU0Cn/5lI5CupZlWaPZtCHfJEbxPsvFN7KssDFz2nBkYCq97LYrrIL4OkiqRAEm7gyLPqp4+04QTUfEFrPtepHSf9WNa+kw7GaHJ7tvFOXB+5nRQbzw+puC26GugW8gxIQgouSAE/TS6uxNYwyXOnG/8nXPrCwcfjMb0rX9bIfh3YDpZCjkSJIOaFEsidzeefZitNphBElwutBNFoXjDlGMlenNizUECPigdLtda7bDnye0Iq6zsuvBLhLc7fDxxc6hwIeD1CAgeoBwlm5nAWtrIGKmOq0HtqV4CzyKPCFo8Qgz0oh2u28DNr8R78w2ErDJXHzQa/i1aJFVKP3vU4DkyNm0BjIpVN3ckGDW/5q8GLlS2yJWc2AgQFn9yHe+NirzJCeTujO2m/vcrRIZDPxggV4/a1oD8T+T2Tp7LMCVPabemZLK27NYMdPqwp3uAkZDaqGQ5qUdPpX6VQ0dzysAnFcdDZPaoe4iaZOKoKcIo7vGleqr0kz1pdUJDDWU5Iqv6KNGF4haBfxKqfNLvIL/ChQrCxUJfaJ2SCGAJMWWhLAXZAZbmEKnF38Cee/BCdefMUBAFH3AMPHg/Q+Idzb1QjmKJfuipI6ZuT6pZKd2ExfDIX5zwXmTlzM9tXpczitlvnHgd/1LwdvN3PjTfQpYoMnM4o3dvQRlKLSvNjAqNq2YgHptMrPj2SE20j/ePiE21j3ZDPwB0AushjpauyjwPCcGPRVt941fHZdiJ1PCOjzODt7nZvCs76xuEfZPTH1HHO5ZkczZdMF0tHO0sToSBu4aleovx7pgH83Cf7bbzK4+ZYldfUnk1TdJAVOg7BVAfL0YIvNtGZgCgg8jLNBrFiY5
x-ms-exchange-transport-forked: True
Content-Type: multipart/alternative; boundary="_000_AM0PR0702MB36035772105916DF3C993792AC469AM0PR0702MB3603_"
MIME-Version: 1.0
X-OriginatorOrg: ericsson.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR0702MB3603.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ec426d16-5079-428c-de51-08d9056613c9
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Apr 2021 08:10:22.5256 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yFjMfFTv7FpOG/jkcgHLkXvnpXc8cOduhWd9s1Z/yL37+j8pluJXVgwIaGS2rDWuNe/dNb8C6qrTIjv0ektvixtFVtEu40owDrVXR4lcr0c=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7843
Archived-At: <https://mailarchive.ietf.org/arch/msg/detnet/tZLGtwmUqUYZPL1DWCh938Suhp4>
Subject: Re: [Detnet] Packet Ordering Function (draft-varga-detnet-pof-00.txt)
X-BeenThere: detnet@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions on Deterministic Networking BoF and Proposed WG <detnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/detnet>, <mailto:detnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/detnet/>
List-Post: <mailto:detnet@ietf.org>
List-Help: <mailto:detnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/detnet>, <mailto:detnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Apr 2021 08:10:31 -0000

Hi Ludovic,

Many thanks for your comments, really appreciated.

Your comments pointed to some characteristics of POF, what may need to be better highlighted in the draft.
- DetNet uses PREOF for service protection and DetNet excludes usage of ECMP (RFC8964). So, any out-of-order
is caused by PREF (more exactly, out-of-order is caused by PEF).
- POF is never a standalone function in DetNet scenarios, there is always a PEF function before POF applied. The
impact of PEF function on the packet flow must be considered when POF is engineered or evaluated. We may
use for the analysis the rather detailed specification of IEEE 802.1CB on Replication and Elimination functions.
- DetNet uses a circular sequence number space, so comparing 2 sequence numbers is always relative.

Regarding your comments:
1, “POFTakeAnyTime"
Yes, reboot of the source (or just a reset of the sequence generation function in PRF) results in a possible sequence
number order (inside the DetNet network) as described in your example. However, the packet flow hits first the PEF,
having a history window. The PEF will discard the packets with sequence number below 1000, so they will not be
processed by the POF.
Section 4.5 in the draft, refers to that impact of PEF with an example:
“   … For example, under normal circumstances the
   difference of sequence number in consecutive packets is bounded due
   to the history window of PEF.”

I think based on your comment, that impact of PEF would be worth to highlight more in the next version of the draft.
Dealing with reset/reboot scenarios is not a trivial task and may need further markers/flags traveling with the packet.
Reset related improvements were discussed in IEEE, you may check here:
https://www.ieee802.org/1/files/public/docs2020/new-varga-FRER-seamless-reset-0320-v02.pdf
https://www.ieee802.org/1/files/public/docs2019/new-varga-FRER-improvements-0719-v01.pdf

2, “POFLastSent update”
Here I do not agree with your conclusion. The POF algorithm never “stucks”. POF maximum impact on a
packet is to delay it for “POFMaxDelay” time. “POFMaxDelay” is engineered to ensure that the packet
is still within its latency budget.
So as in your example: yes, packet103 will be delayed for POFMaxDelay time. Packet104, packet105
etc. (when received in-order) will be delayed less (if at all). All packets remains within the latency budget.
Average latency is somewhat increased but in case of DetNet we do not care about average, the target is
to be within the bounded latency, what is achieved for all packets.

Thanks & Cheers
Bala’zs

From: detnet <detnet-bounces@ietf.org> On Behalf Of Ludovic Thomas
Sent: Wednesday, April 21, 2021 10:53 AM
To: detnet@ietf.org
Subject: Re: [Detnet] Packet Ordering Function (draft-varga-detnet-pof-00.txt)


Hi all, I have two thoughts on it:

=======

1/ what are you thoughts in changing:

"""" [Current version]

The next received packet must be forwarded and the POFLastSent

      updated wheno the POF function was reset OR no packet was received

      for a predefined time ("POFTakeAnyTime").

""""



for



""" [Proposed version]

The next received packet must be forwarded and the POFLastSent

      updated wheno the POF function was reset OR no packet with a seq_num strictly higher (>) than POFLastSent was received

      for a predefined time ("POFTakeAnyTime").

"""

?



Consider the following use-case:



Source has been up for a while, at POF point we received and sent thousand packets such that POFLastSent = 1000

Suddenly the source reboots and the reboot is so fast that the time elapsed between:

               - when packet noted 1000 was sent prior to reboot

               - when packet noted 0 is sent after reboot

is less or equal to the inter-packet spacing. (meaning the time elapsed between 1000 and 0 is about the same or even less than between 999 and 1000).

As such, in any coherent design the time elapsed between 1000 and 0 is less than POFTakeAnyTime.



Now assume POF function receives 1000, 0, 3, 1, [a sequence of out-of order packets], 100, 102, 101, 104, 103 ... again with a spacing between 1000 and 0 that is less than POFTakeAnyTime.



* With the current version, after reception of 1000, timer is started, but reset as soon as 0 is received.

At the end of the day each receiving packet resets the timer before it fires, so POFLastSent stays at 1000 and all packets are forwarded without being reordered until the sequence number reaches 1000 again.



* With the proposed version, neither 0 nor 3, 1, etc resets the timer because their sequence number is <= POFLastSent.

After some time (say just BEFORE packet 100) the timer is fired, POFLastSent is set to 100 and starting from there is the output of POF is back in order.



======



2/ also, the draft states



o  If (seq_num <= POFLastSent + 1)



      *  Then the packet is forwarded and "POFLastSent" is updated

         (POFLastSent = seq_num).



I believe we shouldn't update POFLastSent if the seq_num is <= POFLastSent. In the above example, assume the POFTakeAnyTime is fired just AFTER packet 100. Packet 102 is received, is sent and sets POFLastSent=102.

With the current version, 101 is received, is sent (because seq_num <= POFLastSent + 1) and sets POFLastSent=101, from there the system is stuck because 102 will never be received again, and the following packets will be released after the POFMaxDelay expires.

With the proposed version (no update of POFLastSent if the seq_num is <= POFLastSent), then 101 does not set POFLastSent=101 and the system continues normal operation.



======



My two cents,

Ludovic