Re: [Idr] BGP Message level ACKs

[Jeffrey Haas <[e-mail]>]

> On Mar 10, 2023, at 7:09 AM, Claudio Jeker <[e-mail]> wrote:
> You realize that the output message queue of a BGP process grows to a
> multiple 1000 messages during a full sync. So your suggestion will not
> work. I don't understand this resistance against a new timer.
> Every BGP implementation has some form of timer API implemented and
> extending it should not be that complicated.
> 
> There is a reason why progress is measured as any byte sent during a
> timeperiod. As I exlained already the goal here is not to have something as
> precise as the HoldTimer which is re-triggered by any message. The goal is
> to have something in place for the uncommon case that a system still sends
> messages but refuses to accept messages. It is extra conservative to not
> trigger too early (since that should happen via HoldTimer) or even worse
> trigger when systems get busy because of larger network changes.

Speaking as an implementor, it's not the new timers I find worrisome.  The general idea is largely fine.

I have some lingering concerns about some people having an expectation that sluggish draining of peers will be caught by this mechanism when it won't.  One of the common pathologies that is seen, especially in the face of unidirectional packet loss, is the peer drains but very slowly.  But it will generally drain to the level of permitting the next PDU to be enqueued before the sendholdtimer is likely to expire.

When troubleshooting such issues, typically we'll look at the following bits of state on both sides of the connection:
1. BGP queue levels.
2. BGP packet sent counters (also able to be monitored via SNMP)

These two, observed over time gives one a reasonable BGP PDU/sec metric.  The first item, if it doesn't shrink as evaluated in the SNMP-sense as a gauge, also tells you that there is sufficient churn in the network whether you'll eventually catch up or not.

One clear desire for the sendholdtimer is to catch sessions that block and linger in that state making no progress.  I think it works fine in that case.

The TCP timeout option is potentially useful tool on top of this, specifically in the case of zero-windowing, to detect issues at the TCP layer that may be causing some of the problems.  It is fairly observed that TCP keepalives can help as well, as long as you're comfortable with the behavior in a lossy network situation.  

I have a concern that some operators may want to use the sendholdtimer to catch sluggish draining peers rather than total blockage.  They'll thus turn the time down enough that normal blocked queue behaviors during large convergence events trigger the feature and drop the session.

-- Jeff