Re: [Idr] Existing solutions in enforcing Flow based forwarding can be used by draft-dunbar-idr-5g-edge-compute-app-meta-data

Linda Dunbar <linda.dunbar@futurewei.com> Thu, 03 December 2020 02:51 UTC

Return-Path: <linda.dunbar@futurewei.com>
X-Original-To: idr@ietfa.amsl.com
Delivered-To: idr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 571B23A08B0 for <idr@ietfa.amsl.com>; Wed, 2 Dec 2020 18:51:17 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.988
X-Spam-Level:
X-Spam-Status: No, score=-1.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DC_PNG_UNO_LARGO=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=futurewei.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GpWBf4lWmnBG for <idr@ietfa.amsl.com>; Wed, 2 Dec 2020 18:51:15 -0800 (PST)
Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2136.outbound.protection.outlook.com [40.107.94.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 354BE3A08AB for <idr@ietf.org>; Wed, 2 Dec 2020 18:51:15 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JnNiu2UJZUdsA6uflrMtcDZNDrsUsgAerSZO7lDwwQ/pyhpSHOQOWRLFkHIQ1MFmHRRag01ij6e3/cxvprSfAYwc93cvNkT41fDDFpYqo6fO1sKoDEs9jY6EpDUVNZBHOYV0JgK2UBzNTMOx4IO200uqhv09dUMjwfDKcTy/SPDxUiGE0aUuaOAIPCGYAhhsv5jNSionU7iwn+2/rmrqcg+kFppxSPIhe58hHY7BcR0OQ9ErX2OARg6B2S+q3RCtD2uSF0enk/s58FH6hIeJqGoxH+z7qgnIaY8bpysxvLOKAgAarVzX9KGNCEAYdxUDIFm7ItFPAXTl/M23Ho/kxg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RLVyM1LJHOxOPnZkEQm29O1dQ8s+3jAt863wsCT+59I=; b=TuAgVwmm1/yY+RBgJytXdA+M00D5ZizBI1ZwRFHXPBIwQ5ihzTSdYVU7bx3Y4QEdLmIX/7hxmzQIalaGjg3r58AM2SQAMDO1J7Gu303xdem892qZv2zkx2irdfthmcFhq5jmZGA8/0UMg9BoMsxFhJuSi7AXCHR2v64g3gJuHKufRpHVAFQL7Gj2b0gvXxG4eGsRLMrRLugeRdwFdy0/h19167sod90qfNAvvRVzyqj5EvfVnu3RGJbxv9eMmIh1dyLPl0kfOnwoAtV73xrnVA8nx+DGK8UlPeAyaeQ2+FaaKFlp+MLQFwKTRVnt0YFm5pUQiIzmVy5cM7ndPi3WUw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=futurewei.com; dmarc=pass action=none header.from=futurewei.com; dkim=pass header.d=futurewei.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Futurewei.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RLVyM1LJHOxOPnZkEQm29O1dQ8s+3jAt863wsCT+59I=; b=R6Ywzrnx3Or2u1FQMprWiNpFwzWVUiH9tP2VysbDLFNltn0D2VWpP+ppoAcDkfZBTKnDBt09U7Wh6wH2Na2y3D2cDhFIJHSW9LiUz4tChwm7WQ6AAAki/2bAzP4s7PdS/wfa2xkesqMgwdCK0G9SHvVgJvdTT/3x5idvQdBP7DE=
Received: from SN6PR13MB2334.namprd13.prod.outlook.com (2603:10b6:805:55::16) by SN6PR13MB2270.namprd13.prod.outlook.com (2603:10b6:805:61::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3632.16; Thu, 3 Dec 2020 02:51:13 +0000
Received: from SN6PR13MB2334.namprd13.prod.outlook.com ([fe80::c55c:472b:34c2:c1ac]) by SN6PR13MB2334.namprd13.prod.outlook.com ([fe80::c55c:472b:34c2:c1ac%6]) with mapi id 15.20.3632.009; Thu, 3 Dec 2020 02:51:13 +0000
From: Linda Dunbar <linda.dunbar@futurewei.com>
To: "Acee Lindem (acee)" <acee@cisco.com>, "idr@ietf.org" <idr@ietf.org>
Thread-Topic: Existing solutions in enforcing Flow based forwarding can be used by draft-dunbar-idr-5g-edge-compute-app-meta-data
Thread-Index: AdbIDqowHb3+VOrRTI6SVuOpy6sGLQAum5cAABSIOkA=
Date: Thu, 3 Dec 2020 02:51:13 +0000
Message-ID: <SN6PR13MB2334FE2CFBD40316F34B9B2785F20@SN6PR13MB2334.namprd13.prod.outlook.com>
References: <SN6PR13MB233415994F565A8F8B12682985F40@SN6PR13MB2334.namprd13.prod.outlook.com> <3C8F724B-DBE1-4C70-A055-9CFB822D69A9@cisco.com>
In-Reply-To: <3C8F724B-DBE1-4C70-A055-9CFB822D69A9@cisco.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
authentication-results: cisco.com; dkim=none (message not signed) header.d=none;cisco.com; dmarc=none action=none header.from=futurewei.com;
x-originating-ip: [72.180.73.64]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a1e6952b-5d71-4273-c7a8-08d897364c01
x-ms-traffictypediagnostic: SN6PR13MB2270:
x-microsoft-antispam-prvs: <SN6PR13MB2270D236FEA021514250E5CF85F20@SN6PR13MB2270.namprd13.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 38FR1w+EHHZW6l48W9TVOIiTtrsu6/oA5iVvlw1V/Pxp/Tjsqfa7suLADtL8Zjs6tboAvH3p8sn2M/Xa/7VyU0ui+CSKb6OI7lolkwLo8mDA8GefcmfJ9FsGK4q35enyB1FGtQLHX0zpHtsCF1OAy6s09j2lzzo4So7i2EAS8MLBqpS4RD3j2lG19CrQUD8tWXo4Ydx7fGttKDXTH6i0Ngbcmu2Rrsm+XhSR+QmIni3XElJNEunKijImY/NBhVz+NTYXaIcHl6ONUZOYo7kIMeaqFC7XzQ+gcTxVrfKPOF4ZAocT2LeWMvU2GWPxVRCSZ0ZSHO/J3JdMRoT5Q6Itn8ltIdaiaX9/8RkuY0O7iHJU2jgjXiGAwvA8dsG4LwZXw2ua95WFUmjVf9IoPWbn1g==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR13MB2334.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(39850400004)(366004)(136003)(346002)(396003)(5660300002)(52536014)(71200400001)(66446008)(66556008)(33656002)(66616009)(64756008)(66946007)(66476007)(76116006)(2906002)(8676002)(99936003)(166002)(8936002)(83380400001)(66574015)(966005)(316002)(55016002)(110136005)(44832011)(9686003)(86362001)(478600001)(26005)(186003)(6506007)(53546011)(7696005); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: =?utf-8?B?Vk1rM3pmZUFVVnZZd1Q3bWNiR0lYS1ZQcWFZZ1V1N2dBRGdQTzg0OVk2ZjRP?= =?utf-8?B?dFdVNi9FRnVFUmF6THgvcGRaalhESDhydXYySjVqVnMrdzhkN3ZScGhMZk5s?= =?utf-8?B?MTVtajFtUU1sZ0MvOW5iT0tpWTFtUlFTRDBvQVZ3QlgzYkVFS2NyUmFiSlA3?= =?utf-8?B?NTNkNlVJb0Y4cFlWMmxXcVFCc1l1RHFnMXluRENGL0xzRWR6TnZJWmlrZy9l?= =?utf-8?B?dzNxb1FIbjRnMTNGUWQxYXNESG5LWFhNbEJwUHJzUG9vOHYvclJTdnRqWXJS?= =?utf-8?B?QzYyWFpESzduelpRVzdHRHZxdk1wVlNhVkZMUlMrV2t3b21uUUEyWml3VTRI?= =?utf-8?B?cERBZkdhVUR6eE54NG9iaENad1BhK3pMelk5Nm11NFpLZkZ6akdxUlFaZ3JM?= =?utf-8?B?ZlBGTk1vaDdKUXRUaDdBTElWQlFNdnhUY1BlZFdySXZSYmlrQVNscGVsQWcw?= =?utf-8?B?MXk2N1dwLzZwYzFLbFpLQkNTUWZyaERmeWR1NlJzS2pqNHRTV1QvQ09IYXJs?= =?utf-8?B?UWQrelJZeFlyMEN2cWtadElGT01YNjJGcmh3SU9UT1QzSFRLMUxraE90ZFYv?= =?utf-8?B?L3RqWHRFYjUrU1ZnMHBqTFNYUkg3cDUxQk9wUjQvbXZoSVZkeC8wS1lURmw0?= =?utf-8?B?eCtCQko3UlRUbzZibW1NTWZzVEhuL2ZXL3Fva3ZWdnJtdzFwaDU3Wi83ekpv?= =?utf-8?B?eFRBZ1dtVENYOXgzZjFKSHIzcEpyellkcXBOdDFGeERqNjc4djVoZElybkxX?= =?utf-8?B?ZHRwQUEyckFZVE5UYWw5NThNTmFWRStJRm5hRjhYbmdFV1NZYTVvUlY1UEhv?= =?utf-8?B?aGhQSVExQjZUNjRNVmtNVFgvVmFhQmxGN0pCZGhvK0sxS2pEUlhIMnFEM3JY?= =?utf-8?B?UEVyNUNiWGp6M3g4MmU3bzI5T29PT3gxTzhoYW1MeGNUdnlxUGdDVlIyTHpM?= =?utf-8?B?S3c0a1l1RUY4RlVUNDFYWVVKaTRJTWxISFNMKzl6OWJLbnNETmQxL0l5dGJm?= =?utf-8?B?WG4wb25sc0JxTThMVndtNDdrY3h3a1pnVWNITEI2L2hjQXIxR211dlZoSGdG?= =?utf-8?B?SnhuQm9TSkpWejlSazc3cnU5Zm5rYzV4c2dNVUpTY0E2bHhDSjhWZjhMdHJI?= =?utf-8?B?UUZnRjZqaVNyNFBQTWU5bTRjMC9HRnk5SzVvU3RtRXRETXRaNEkwM1FuQ2VS?= =?utf-8?B?YnZRRDVidjVneG9mOG9YNGNwaGZJbFY1ZUJPTmdNVEQ5c2pQbVJtOUZ0dWwv?= =?utf-8?B?UTRTM1U1RGFzQU1rb0hSRktQVHRxczZjRWJPUnZ1Q29FaEw4eUFuUTFOYmls?= =?utf-8?Q?XaFX8uFJAQQYM=3D?=
x-ms-exchange-transport-forked: True
Content-Type: multipart/related; boundary="_004_SN6PR13MB2334FE2CFBD40316F34B9B2785F20SN6PR13MB2334namp_"; type="multipart/alternative"
MIME-Version: 1.0
X-OriginatorOrg: Futurewei.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR13MB2334.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a1e6952b-5d71-4273-c7a8-08d897364c01
X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Dec 2020 02:51:13.0899 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 0fee8ff2-a3b2-4018-9c75-3a1d5591fedc
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: vKXr/ARXFg48ZGvYcaTSdgzNBmqcfZAPu+qQlF6aDYaQeLivNGEf4BisayUI/a0re+W8m93QZ1ThW9s+d1fwSQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR13MB2270
Archived-At: <https://mailarchive.ietf.org/arch/msg/idr/OKavXJMgU1NxBwC-NqDSHmcrvHc>
Subject: Re: [Idr] Existing solutions in enforcing Flow based forwarding can be used by draft-dunbar-idr-5g-edge-compute-app-meta-data
X-BeenThere: idr@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Inter-Domain Routing <idr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idr>, <mailto:idr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idr/>
List-Post: <mailto:idr@ietf.org>
List-Help: <mailto:idr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idr>, <mailto:idr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 03 Dec 2020 02:51:18 -0000

Acee,

In the figure below: 3 App servers for S1 (ANYCAST address) are attached to R1, R2 and R3 respectively. R1/R2/R3 are called the App Server Egress routers In the draft-dunbar-idr-5g-edge-compute-app-meta-data.

Ra/Rb are the ingress routers to Local Data Network (the N6 interface in 3GPP spec). There can be many hops between  Ingress routers and Egress routers. They are similar to EVPN’s edge nodes.
The proposed NLRI App-Meta-data SubTLV is for App Server Egress routers to advertise the LoadIndex & SitePreference & SiteCapacity  to the ingress routers.
When Ra determines R1 is the optimal location for a flow from UE-1 to S1, Ra tunnels the packets from UE1 to R1. So all the immediate nodes don’t see S1.

Answers to your specific question are inserted below:


[cid:image002.png@01D6C8EC.DDAD0870]


From: Acee Lindem (acee) <acee@cisco.com>
Sent: Wednesday, December 2, 2020 3:35 PM
To: Linda Dunbar <linda.dunbar@futurewei.com>om>; idr@ietf.org
Subject: Re: Existing solutions in enforcing Flow based forwarding can be used by draft-dunbar-idr-5g-edge-compute-app-meta-data

Hi Linda,

There are a couple limitations here that need to be covered in the draft. The first is that you are assuming the anycast servers are all attached to a router one hop away (or at least on a path where the first hop will always on the shortest path to the same instance of an anycast server).
[Linda] There can be many hops between Ra and R1/R2/R3. Ra tunnels the packets to R1, all the immediate nodes only see unicast address of R1.

Otherwise, your assumption of the routing decision being made solely in the 5G site router doesn’t hold.
[Linda] The path selection is by the router adjacent to the 5G PSA (part of the UPF)
Second, when there is a handover from the Site A router to the Site B router, how does the Site B router know which instance of the anycast server the UE was bound to on the Site A router? I guess it doesn’t without some design.
[Linda] https://datatracker.ietf.org/doc/draft-dunbar-6man-5g-edge-compute-sticky-service/ describe the approach for ingress node to stick one flow to the same location even after the UE anchored to a new PSA (i.e. moved to a new 5G site).

Please let me know if I have answered all your concerns.

Thanks, Linda

Thanks,
Acee

From: Linda Dunbar <linda.dunbar@futurewei.com<mailto:linda.dunbar@futurewei.com>>
Date: Tuesday, December 1, 2020 at 1:21 PM
To: Acee Lindem <acee@cisco.com<mailto:acee@cisco.com>>, IDR List <idr@ietf.org<mailto:idr@ietf.org>>
Subject: Existing solutions in enforcing Flow based forwarding can be used by draft-dunbar-idr-5g-edge-compute-app-meta-data

Acee,

You asked a question on how to enforce one flow to be nailed towards the same location for an ANYCAST address during the IETF 109 IDR Friday session.
Here are some links showing that the commercial routers already support the feature, a.k.a. Flow Affinity, or Flow-based load balancing.
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/lanswitch/configuration/xe-3s/lanswitch-xe-3s-book/lnsw-flow-portchannel-load.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.cisco.com%2Fc%2Fen%2Fus%2Ftd%2Fdocs%2Fios-xml%2Fios%2Flanswitch%2Fconfiguration%2Fxe-3s%2Flanswitch-xe-3s-book%2Flnsw-flow-portchannel-load.html&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C0b90845aa45f400e455c08d8970a5394%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637425417898549584%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VBY6rgIaAD9GlJvHH2ru1Dvd6SEy8jdnH5pbRzjqiSg%3D&reserved=0>
https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/data-flow-affinity-edit-chassis.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.juniper.net%2Fdocumentation%2Fen_US%2Fjunos%2Ftopics%2Freference%2Fconfiguration-statement%2Fdata-flow-affinity-edit-chassis.html&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C0b90845aa45f400e455c08d8970a5394%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637425417898559538%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RdIFXWipGwCOAz5pwFzbFeKdbolPfW1TRP2BdD1v6nY%3D&reserved=0>

draft-dunbar-idr-5g-edge-compute-app-meta-data states that the ingress node, i.e. the router Ra/Rb which are adjacent to 5G UPF (or Packet Session anchor point-PSA), can use Flow ID (in IPv6 header) or UDP/TCP port number combined with Source Address in IPv4  to enforce packets in one flow being placed in one tunnel to one Egress router, such as R1, R2, or R3 in the figure below.  All other nodes in the network don’t need to take any extra action.
Does it address your concern?

+--+
|UE|---\+---------+                 +------------------+
+--+    |  5G     |    +---------+  |   S1: aa08::4450 |
+--+    | Site +--++---+         +----+                |
|UE|----|  A   |PSA| Ra|         | R1 | S2: aa08::4460 |
+--+    |      +---+---+         +----+                |
  +---+    |         |  |           |  |   S3: aa08::4470 |
  |UE1|---/+---------+  |           |  +------------------+
  +---+                 |IP Network |       L-DN1
                     |(3GPP N6)  |
  |                  |           |  +------------------+
  | UE1              |           |  |  S1: aa08::4450  |
  | moves to         |          +----+                 |
  | Site B           |          | R3 | S2: aa08::4460  |
  v                  |          +----+                 |
                     |           |  |  S3: aa08::4470  |
                     |           |  +------------------+
                     |           |      L-DN3
+--+                 |           |
|UE|---\+---------+  |           |  +------------------+
+--+    |  5G     |  |           |  |  S1: aa08::4450  |
+--+    | Site +--++-+--+        +----+                |
|UE|----|  B   |PSA| Rb |        | R2 | S2: aa08::4460 |
+--+    |      +--++----+        +----+                |
+--+    |         |  +-----------+  |  S3: aa08::4470  |
|UE|---/+---------+                 +------------------+
+--+                                     L-DN2
Figure 1: App Servers in different edge DCs
Thank you very much
Linda Dunbar