Re: [Idr] WG LC for draft-ietf-idr-bgp-sendholdtimer-03 (3/23 to 4/12/2024)

[Ben Maddison <[e-mail]>]

Hi Jeff,

On 03/25, Jeffrey Haas wrote:
> Ben,
> 
> > On Mar 25, 2024, at 6:44 AM, Ben Maddison <benm=[e-mail]> wrote:
> > The only remaining issue, which I believe warrants further discussion in
> > the WG, is the guidance that BGP speakers should send a NOTIFICATION in
> > response to the SendHoldTimer_Expires event.
> > 
> > Although I appreciate that for the purposes of internal consistency it
> > is desirable that the FSM mandates a consistent set of steps to take
> > when tearing down a session, I think in this case it results in further
> > contradiction.
> > 
> > Consider the possible outcomes for an implementation responding to a
> > SendHoldTimer_Expires event by sending a NOTIFICATION to its peer, as
> > written currently. Either:
> > 
> > 1. The local speaker waits indefinitely for confirmation that the
> >   message has been sent on the socket, delaying the closure of the TCP
> >   connection, and defeating the objective of the timer entirely; or
> > 
> > 2. The local speaker successfully sends the message to its peer, and
> >   finds itself in the contradictory position of having successfully
> >   sent a message to the peer, whilst handling a condition that has
> >   arisen due to its supposed inability to do exactly that!
> > 
> > I believe that the intention here is that the local speaker performs its
> > internal bookkeeping and event logging using the "Send Hold Timer
> > Expired" error code, but only attempts to send the NOTIFICATION to its
> > peer if it can do so without blocking the remainder of the session
> > clean-up.
> 
> The intention overlaps the text I did for the BFD cease subcode, now
> RFC 9384.  In some circumstances, the connection may effectively be
> gone anyway, but we're in need of having diagnostic information as to
> why.  In the case of that RFC, we're not guaranteed to be send blocked
> but the TCP connection might otherwise have reason to be gone by that
> point anyway.
> 
> As you note in 2 above, successfully sending the notification would be
> perverse since we should only enter this situation because we're send
> blocked.
> 
> When you examine the remainder of the sendholdtimer text in section
> 3.3, it follows the usual pattern of sending a notification where we
> send it adn then tear down the tcp connection anyway.  The point I
> think you're highlighting is whether "send a message" should be
> considered a synchronous operation or not - see at least one prior
> comment I'd left in the thread for this feature suggesting that
> shouldn't be the case.
> 
> The desire, as you note above, is that all of the work we'd have done
> if we had sent the message on the wire even if we can't even
> successfully enqueue the message to tcp.  The implementation would do
> its diagnostics, track last sent code/subcode, etc.  

Agreed, we're on the same page.

I think the fundamental issue with the text (and other similar parts or
RFC 4271) is that sending the NOTIFICATION is mandatory, and must
(obviously) happen before the socket is closed, leading to the naive
interpretation that the send must complete before the remainder of the
actions can proceed.

Expecting future readers to read this as "oh, I should enqueue this to
my write-thread, drop the handle (or future, or whatever), and proceed
to kill the socket" is asking too much.

> Is there any particular text you'd want to indicate the desire is that
> this should be done asynchronously in spite of the fact that nowhere
> else in the BGP FSM text we discuss how such enqueueing is done?
> 
> > Similarly, it seems unnecessary and confusing to restart the timer when
> > a NOTIFICATION is sent.
> 
> That's a reasonable exception case.  
> 
> > 
> > I have attempted some wording changes to address this, and submitted
> > them for review in [1]. I would be glad to hear the thoughts of
> > participants that are more practised in FSM surgery on this.
> 
> The current pull 4 diff is I think overly proscriptive, but is
> iterating in the right direction.

I attempted to stay clear of phrases that are overloaded with
implementation- or language-specific behaviour. The intent is to convey
that actually sending the NOTIFICATION should be considered optional,
and should not be attempted at the expense of promptly killing the
connection.

I'm by no means wedded to the specific language if anyone has other
suggestions?

Cheers,

Ben