Re: [Idr] Fwd: New Version Notification for draft-spaghetti-idr-bgp-sendholdtimer-05.txt

[Jeffrey Haas <[e-mail]>]

Robert,

On Thu, Aug 04, 2022 at 06:23:58PM +0200, Robert Raszuk wrote:
> So we are trying to drop a session towards the BGP peer who does not
> respect basic RFC4271 obligation to drop the session when HOLD TIMER
> expires. That to me is already marginal (read noise corner case).

>From the perspective of the upstream BGP Speaker, all we know is data is not
being consumed.  

>From the perspective of the downstream BGP Speaker, any number of things
could be going on:
- BGP should know that it has data and simply isn't consuming it.  (A bug in BGP.)[1]
- BGP isn't aware that that it has pending data. (A bug in the TCP stack/OS.)
- A bug exists in the TCP state between two systems and the window has
  contracted out of the control of the application.  BGP can't do anything.
  (A bug in the TCP stack, or an attack against the TCP protocol.)

The proposal at hand is that without regard as to why the downstream BGP
Speaker has gotten into this situation, there's a desire to terminate the
session.

> Then we worry that LINUX TCP stack may have buggy implementation of
> TCP_USER_TIMEOUT and instead of fixing it locally we dismiss this option.

I think obsessing over a specific current implementation is premature for an
Internet-Draft.

> Isn't this going a bit too far ? If the problem is real why don't we solve
> it in a more elegant and broader way (broader in a sense to detect more
> issues) by defining a new BGP Message - BGP PING Message. Sort of BGP level
> heartbeat.

See discussion below.

> Difference from KEEPALIVE messages would be that apparently implementations
> and RFC4271 decouple completely KEEPALIVE receiving from KEEPALIVE MSG and
> tightening them may be a bad idea.
> 
> Long run we could use BGP PING and BFD and retire KEEPALIVEs.

As much as I'm a fan of BFD, it is unlikely to become pervasive for all
situations where BGP gets used.  Also, BFD is insufficiently coupled to the
BGP session state.  By analogy, LDP still has keepalives in its TCP session
even though it has a separate hello layer.

> I shared this offline with some IDR WG members .. now just sharing with you
> all ... It is still different from BGP OPERATIONAL MESSAGE that if we think
> the issue of a stuck/buggy BGP peer is real perhaps could deserve a small
> draft to document it.

I don't think think the op message helps us here either.  The whole point is
the connection is unidirectionally stalled and the upstream wants to take
action on that.

-----

To throw a related example into the thread, my employer's implementation has
had hidden CLI knobs to throw various things into the BGP Marker field over
the years.  Such things have included timestamps or checksums and have been
useful in a unidirectional direction to help troubleshoot things during
development.

Your "ping" observation is something I'd reframe as "how do we update the
protocol to help each side to know things are being bidirectionally
consumed".  The effective state desired is:
- My current sequence number.
- Your last handled sequence number.

Yes, this resembles all of our favorite moving window protocols. :-)

Where could this go into BGP?  Ideally, a clean update to the PDU format.
As a hack?  As two uint32's embedded in the Marker, capability negotiated.

The better question, does this help the situation in question?  
In my opinion, no.

In a "ping" or "bidirectional check" mode, the general idea becomes, "I sent
you seq# X and I want to eventually see this come back to me".

To get the echo reply, the echo must first work its way from the sender's
TCP send buffer, across the network, through the receiver's TCP receive
buffer, into the BGP, then the full loop again through their send and your
receive buffers.  You'd also require a message to message to piggyback the
reply to.  When there are pending updates, it's easy.  Otherwise, you could
force another message like keepalive.

(Yes, arguably you could do a less frequent "ping", but now you're also
adding another set of optional timer procedures.)

By analogy to BFD, you're getting the result of BFD Echo mode rather than
BFD Async mode - and it's likely far longer than is desired for this use case.

-----

Side reading, some thinking on timestamping BGP updates:
https://datatracker.ietf.org/doc/html/draft-litkowski-idr-bgp-timestamp-00



-- Jeff

[1] Discussions with BGP developers over the years about various upgrade
scenarios often comes back to the fact that the holdtimer is unidirectional.
During an upgrade where the TCP and BGP RIB state is maintained, it's useful
to simply pause consuming incoming routing state and do nothing other than
send your keepalives.  The challenge for this discussion is that this
"feature" is indistinguishable from bugs observed for sessions that are
stuck for too long.