Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer - Feedback requested
tom petch <ietfc@btconnect.com> Mon, 26 April 2021 16:08 UTC
Return-Path: <ietfc@btconnect.com>
X-Original-To: idr@ietfa.amsl.com
Delivered-To: idr@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3EC6E3A26B1 for <idr@ietfa.amsl.com>; Mon, 26 Apr 2021 09:08:07 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=btconnect.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3Qmese4DkF_W for <idr@ietfa.amsl.com>; Mon, 26 Apr 2021 09:08:02 -0700 (PDT)
Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2091.outbound.protection.outlook.com [40.107.20.91]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 99C323A26AF for <idr@ietf.org>; Mon, 26 Apr 2021 09:08:02 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VRVMqVkppYum685LWByMcyIGVQGye/CFKiQmUfPOypxwE1egIsPYHN+kgXdMqwXgmVXZgGBl8ioLGRaARcgSl5Cp15LDYsuOTYYhtvPyadD1GkmXku38zUeip29ifk1EI7IGMTKkOm0UQzaRFmQDXhcbWf9819BDV9mDYidlw9hSh2Gem+6Q6O7wiNtRP2zOdV1UwQn6ozqEy+w6pukHkzCeD72TYjnKH60ozjTFVZ5WI4ndfyOKT1dk3OLge/Ff6hf8DachQI5SPXmg0PkHIm+CqfJdtizTh1Q+GXJGB6eMlpAF1Z+zOjdTfbU2uESL6tznGAyHrAkfaXIJFhFTRw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pT7CO13DmhAgKUFcm8r2s8A/qt/JjWj2vTet+5TQeKg=; b=XO9giYbarrBffjBrRxDuHGshEV1oHW070xnivKgY9dZiE4IKkM0eofMIlUoux2+ARGDtN5PiFzF4UogaVTyd34Sq7FPI8LsXG70D/opowrO/awgcvdxfv9T5ABqJjewwDECzx5iryZ/cG8jD4uE6ChdG8B7LrS07TZwrqgRmTz5IEtnyIQSpLMDaJUyug4JooxcjZ6rNfMh4HQ+wfAUX5P+pK5jEoIM08bLmp6YqZc5XUXViCAVFqwIGknw23JJIXxykI8yLV8rvxlQjSN4g2Tr2DhK6ml9cOA7SJSMM7V7uL/FSNN/Zc0jx9fyRkV7qpnQRvcTwlTX3Yu+qMN/GIQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=btconnect.com; dmarc=pass action=none header.from=btconnect.com; dkim=pass header.d=btconnect.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=btconnect.onmicrosoft.com; s=selector2-btconnect-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pT7CO13DmhAgKUFcm8r2s8A/qt/JjWj2vTet+5TQeKg=; b=IrehqvZdQX+diwKMRuwRN2Jlm/Bfhd+TNFHryvwyq+3n2BQB2qkPR3xp4a6/m1cmzUVuB9MEGcek0o98VPl4e4c0sDVUevC+tuUknJLeFh4Ns01UKVciNck/SUmhwCVqyEtKHZ8FZ6TlooymkgFjaHnyy1Nd9tO1RvJsCU4dIlk=
Received: from AM7PR07MB6248.eurprd07.prod.outlook.com (2603:10a6:20b:134::11) by AM7PR07MB6706.eurprd07.prod.outlook.com (2603:10a6:20b:1a6::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4087.21; Mon, 26 Apr 2021 16:07:59 +0000
Received: from AM7PR07MB6248.eurprd07.prod.outlook.com ([fe80::543d:497d:ba3f:5576]) by AM7PR07MB6248.eurprd07.prod.outlook.com ([fe80::543d:497d:ba3f:5576%4]) with mapi id 15.20.4087.023; Mon, 26 Apr 2021 16:07:59 +0000
From: tom petch <ietfc@btconnect.com>
To: Jeffrey Haas <jhaas@pfrc.org>, Ben Cox <ben=40benjojo.co.uk@dmarc.ietf.org>
CC: "idr@ietf.org" <idr@ietf.org>
Thread-Topic: [Idr] draft-spaghetti-idr-bgp-sendholdtimer - Feedback requested
Thread-Index: AQHXNeyWuL85GQc5306CuZDl5UxM3KrCogwAgARYeIo=
Date: Mon, 26 Apr 2021 16:07:59 +0000
Message-ID: <AM7PR07MB62481E54466B9E2840F867DDA0429@AM7PR07MB6248.eurprd07.prod.outlook.com>
References: <CAL=9YSVy+mvxvAv+maxkUSzPbe0bfnUy-XJJTtcVhi3S3bm=WQ@mail.gmail.com>, <20210423212348.GB19004@pfrc.org>
In-Reply-To: <20210423212348.GB19004@pfrc.org>
Accept-Language: en-GB, en-US
Content-Language: en-GB
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: pfrc.org; dkim=none (message not signed) header.d=none;pfrc.org; dmarc=none action=none header.from=btconnect.com;
x-originating-ip: [86.143.250.49]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: c53f9af1-a497-40f5-ae32-08d908cd7624
x-ms-traffictypediagnostic: AM7PR07MB6706:
x-microsoft-antispam-prvs: <AM7PR07MB67068DDAA0189648470B164AA0429@AM7PR07MB6706.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 6dUn4F6S7WP7FRlMpGb/0pgFgUt9NR5Xauqa5JTOEjAlNTNEqsGA2nGrXo4VvdaU2IRCqFgabHlDpn7uDrj1XvxIdBRcIhijK4lhfF6zenVJv+MqJbsv4m+I+xeirG2SizIenEJubfgX6Hns3g3kJjR+oqaSlEn0WfdcjyD4FwlkKuMjTO5ISolbeGi0ijNhXvVBrxSuQhmMtXkRtwnVjAzlqELRTuH6POluSy3cO5sBaCRm8A/VY4gPSetpSSgKX7Qf2EZVrZ4FbHxKJ+YPezTHLMXr40ZSsB3mLHgOsmT0hqX7cSlmb5fLgKM5wYQpmApjyl1y6umbNLuX5pCARmfExDpKe4V0yK7z0Sb0YnbZ9a0160nY4E+AJDVH1jRk3YqDKP16jqFe+p3FST/jaCxAPC1MkWtzUFb9OlueuuISMfjo0FH33P7VvPI2JUhhCkwaCLfMclzjW5mLI4efdk+ScumE1vqVpU1DRy+60oEKwBlOp2VYQKQi1oRu1leerKNC4fiFKxgJoqTlHR+hihVQH46P8w/taW0nme9jLlQYQqdr1hG/kbZbtwXhEIbj+GM7opTLXxm34vzeWoqsuASOAmUTjfo1gRkk8hAfVqiTWdRv1i8apK76yEcQ8Ukg25/7M1NV0UvJ2pL1q8C10UW8RSW5TKk01ZtVMny/DGk=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM7PR07MB6248.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(39860400002)(136003)(376002)(396003)(366004)(346002)(2906002)(33656002)(478600001)(9686003)(5660300002)(966005)(55016002)(71200400001)(66574015)(8936002)(186003)(83380400001)(8676002)(26005)(66556008)(91956017)(64756008)(52536014)(66946007)(66446008)(66476007)(316002)(122000001)(110136005)(38100700002)(86362001)(4326008)(7696005)(6506007)(76116006); DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: +TLN4knE4NNNY6aGk0I3PpZB5REBPm2qmjIatO0RRqIoqCWNrmmUVSASYJ9jUaKZeiSU2CvdTXeT/LB1ABuFfVWknNCp/Sfv1xGJHqHf2w7IcMmWqSoBeDNQZz51typbUcSRSnVMZHESFt3p+c1T0XZOJaY18zrLVJ/xPFFHBBvHzK9i0rDLk4KERJKKC+Gsp4iE587DgrlzBMWhMOZpjlElNujSgg/c68xeoTolam4NlF+X+2ksKV0dVeb6StncV9lZEapQPYg5AHl1aoPYBI798j4UhMAJ4idZRGjAbXjbIhrdj5EQUEEt/eco4iEKJJvebQ7Xk5YQWgejmVhIzNZvGNhqp+C5od3x1gW+sEEafZ7d+eEEciiMgBmAvR1PbU7QA1IE63a7rYDN2FIzFnTLkueumJV3wXebzG33HEPUFa6aX2nhuMiapJYyGL3DMBudMchVIQJggMKbUMIZDhS2DhxwdAiNXfrCGumft0O8MUDH8Ojwr8XBQYnF5O1JUYUluh+p9MC7CGRGkRQvrN4h7jTgzXBB1BdwGZxYca/49A+iaAmsDUpnCDaLdr1JQK7qY/MXr551uXrPBXPnFHz32+RtjeTZ9V2Jy5pVeLT2lgYQHPTyomoeD3rLnLA3LDiyLc5GNS0XOCKpcGauXzK1cz3P4O2YOzNlL9iHq52trxB/CLtbUn95CttEAJw1hjePFyVomiOBSdIOvA2aTKywacvuPwxdNtRzZ/KfjSFkCGV3dKXxhWcjCaraEoS/zwzgOKrzscHVwU8ctEzJ1NMbSDSpoBDyKuFIzaDci3Pp4XiKXynPFeNehFveqve7AfPOv0fwSBiZIyfH6F7ZNVKH8oOn++K1Ybs19Mxl/BFQsdodG6//emuYgJuHzVcBGVhYsJtluCwkB0jh/NEywX+m3i9q62nUVUlzmswdgtG+g5y3La1+PeF0sgUsht4Z7y2N7qXPUpBPXloKWLl5OZsJvo9BwJFiQED3Y4pRUMjj2QAFEewJqO4rGqD7P/uIb9lokpcZSW463K0YURm9wIt3NY2y9jfZvio8oKXkG3ySqDAVlo+mqOfhLrpuT4fJE72m1W5SZOpgqqlP9fuB3A5nrWnCLq5ShFHa0BU05lrpAN/1jkraYNxhmC/9rRgiollUEwE+0TyAB9Ua+4kL8WLtf5k0r50C4mgIqULDLdHi+3aIww9VVNUHu/vT/hleMsYemAY2UdehV9rVcUP8yEftUSx8O0pmIi2tE2WykJ+x5ZMNbFbDv1TwRA+5V9grIukKXOgkEwEPlczsAsLl+hSW0CMjBW7MVr2bbWXg1vM=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: btconnect.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM7PR07MB6248.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c53f9af1-a497-40f5-ae32-08d908cd7624
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Apr 2021 16:07:59.1655 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: cf8853ed-96e5-465b-9185-806bfe185e30
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: r4n/jAQqaygcbDv7Vx5DYrrOVlMcVozRHHf2Ws3F69p0EI2ncOMIZglvndUB6GEAgrfCiYVJkQOiGdZoNbUScg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR07MB6706
Archived-At: <https://mailarchive.ietf.org/arch/msg/idr/UAwm0ODNIFGoQxro_NDzwIaw7qI>
Subject: Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer - Feedback requested
X-BeenThere: idr@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Inter-Domain Routing <idr.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idr>, <mailto:idr-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idr/>
List-Post: <mailto:idr@ietf.org>
List-Help: <mailto:idr-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idr>, <mailto:idr-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 26 Apr 2021 16:08:07 -0000
From: Idr <idr-bounces@ietf.org> on behalf of Jeffrey Haas <jhaas@pfrc.org> Sent: 23 April 2021 22:23 On Tue, Apr 20, 2021 at 02:47:35PM +0100, Ben Cox wrote: > We submitted our first draft today ( > https://datatracker.ietf.org/doc/draft-spaghetti-idr-bgp-sendholdtimer/ > ) and we are looking for feedback knowing that it is not complete but > is likely in a state for some discussion. > > Since this draft tweaks the BGP FSM, we would like to make sure that > it's done correctly and so we are soliciting help from this working > group. As someone who spent a lot of time working with the editors of what became RFC 4271, you're going to find there's a lot of corner cases in the FSM to deal with. :-) We can work on the text once it's clearer what we should do. <tp> After which we will likely want to change the specification:-) This I-D will cause extensive updates to s.8.2.2 about when the sendholdtimer is started, when it is stopped, when it is cleared. Since this is to do with TCP, then the starting point for the specification would seem to be as soon as the TCP session is established ie once the FSM moves out of Idle state; but even that will not preclude a timer expiry in the Idle state so all states are affected. You have six states and 28 events each of which have to be considered for each state and the effect of some events is affected by what else is happening, e.g. what other timers are running and by how the state was entered, which is often the same thing. So, yes, there are a lot of corner cases and then there are all the points that Jeff so accurately describes below.. This is a substantial change to the core routing protocol. Tom Petch . A number of stream of thought comments about the topic covered by the draft: I don't object quite as much as Jakob does about the issues of claiming implementations that are exhibiting a stuck session as contributing to stale routes and blackholes in the Internet. A simple truth of BGP as a hop-by-hop protocol is that if you can't distribute your updates, the downstreams are operating on an incorrect version of reality. That said, the problem statement isn't really crisp enough I'd want to make such a large claim. TCP applications will exhibit zero-windowing naturally over the course of a session's lifetime. This typically will occur when there's some level of backpressure on the application due to I/O issues or even CPU. While these situations aren't desirable since they signal congestion in the application, congestion SHALL happen.[1] Most of the headaches I'd discusss for the problem this draft attempts to solve revolve around the consequences of this statement in section 2 of the draft: : Generally BGP implementation have no visibility into lower-layer : subsystems such as TCP or the peer's current Receive Window. : Therefor this document banks on BGP implementations being able to : detect an inability to push more data to the remote peer, at which : point the SendHoldTimer starts. It is quite true that it is difficult to get a sense of what's going on when you have simple socket (for example) backpressure. What I believe an application really wants to know for some sort of send-hold-timer is the following: - A session's TCP window has reached zero availability at a given sequence number. - It stays at that sequence number with zero availability for "too long". Typical POSIX socket APIs won't give you this kind of certitude. The condition, as written in the draft, effectively manifests as a write() to the socket that fails. But how does it fail: - Is it because there is no space in the socket to accept data? - Is it because there are no local resources (e.g. mbufs) to push the next bit of data? - Is it because a kernel timer hasn't triggered a bit of TCP behavior to try to move some data? - Is it because we had blocked and we're waiting for select/kevent to fire and let us know that there's space - and it simply hasn't bothered even though there's space in the window? - Is it because a low-watermark feature is preventing the socket from being ready even though the window has advanced? - Is it because the session is draining VERY.... SLOWLY... and the kernel won't wake us up even though the window is no longer zero - just not big enough? (A version of the prior issue.) Some things that can cause zero windowing may be worth keeping in mind to decide how bad they can be and what an implementation may want to do about it: - Perhaps it's doing a live-upgrade and needs to wedge the incoming session to do so? (ISSU, e.g.) - It might be configured with incoming session prioritization, and your socket isn't important enough by policy. (Corrollary: Not all routes are created equal.) - It may be behind in servicing the socket. For example, too much work, perhaps due to a reconfiguration event that will resolve at some point. - A firewall may be interfering with TCP and the receiver is actually fine. Finally, I want to highlight something rather critical: Just because you reset your session, in some of these pathologies the router you're resetting your session with may not withdraw your routes downstream fast enough. This is the stale situation highlighted in the draft. The distinction is that sometimes thrashing makes it worse. If you're going to disconnect from such a peer, you had better have enough routing diversity to survive disconnecting from it. You also better be prepared to keep the session down for some time. Keeping the session down is not addressed in the draft. -- Jeff [1] When I give presentations discussing BGP, I tend to note that BGP is for the most part a simple protocol with a lot of optional features... and the hard part is making it scale. _______________________________________________ Idr mailing list Idr@ietf.org https://www.ietf.org/mailman/listinfo/idr
- [Idr] draft-spaghetti-idr-bgp-sendholdtimer - Fee… Ben Cox
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Job Snijders
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… William McCall
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Enke Chen
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Brian Dickson
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Enke Chen
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Robert Raszuk
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Robert Raszuk
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Robert Raszuk
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… UTTARO, JAMES
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… tom petch
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… tom petch
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… UTTARO, JAMES
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… heasley
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Robert Raszuk
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Adam Chappell
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jeffrey Haas
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… tom petch
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Robert Raszuk
- Re: [Idr] draft-spaghetti-idr-bgp-sendholdtimer -… Jakob Heitz (jheitz)