Re: [Lsr] Why only a congestion-avoidance algorithm on the sender isn't enough

Mitchell Erblich <> Mon, 04 May 2020 02:49 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 481553A0AF8 for <>; Sun, 3 May 2020 19:49:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key); domainkeys=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 3Fl9tMYqOo64 for <>; Sun, 3 May 2020 19:49:08 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 7F1F73A08B9 for <>; Sun, 3 May 2020 19:49:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=dk12062016; t=1588560548; bh=xY2at8Q7HVCWIzz8LL3jFR67he6SkgaVodzG xSZF0M0=; h=Received:Content-Type:Mime-Version:Subject:From: In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id: References:To:X-Mailer:X-ELNK-Trace:X-Originating-IP; b=AfDsT3oY47 pNyMXoZ1x8I/esRx+Y+ojtBwhlm4fRvGUl4wPQRs9th7qGlihPE1Hw2QwYaOk9NDCjP X3ae7hGl/SltYKSS7xvsUL1BdjSMH5FKl3YO7dl/XGJjb35W59uzpKLkNQWPvg9J+ht ic/YCFqluiAJOJqbJtt5vX3HaMefFRP64R5l/sOMZBzJDZeqDX9XJpMMC3DTRFJz3CD jF8rwxkHX3SJunAGVNzzjdubXtTySQJ8TQc60GVvK+haBp7oo9yaIeDrvdAYAriypkA Z1UgZiSJfz/xHUJ5bvHPb07YPZvx2jaOFkrns4w8KEKTVJE4mfE156HQ3KX3quLQ==
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk12062016;; b=HHXoczJAzW6G+oK41kSFbvFOP4O0wENasohG8DJC8GQKNL8hSn6FzTrNj6svMUk70kyWGzI01hOtONgXhKAxzlH3w+JKhHiJ8/7nmmjsbrHFjqlgAbm4+eptpxSoscpe2Pnli1RMt5IM/16WUBxpDb7MZ2SOR2jNQKQXKPEWTLwZAPzLoiKRPmVPoSfvIhs/jkuZAjwPEplG0MWCiIpxGs+Fgh4oyNLxPtAn12nYNmdwh7L5MTTd9ZBP70z95djZO0/OfqvmJwVQZAveKJBrCapARwjnVIm80WKvVsttwqmm6vQLNNXbUJlCUbgKXLSOjNF+YM3HK9LO6wltY3Xc8Q==; h=Received:Content-Type:Mime-Version:Subject:From:In-Reply-To:Date:Cc:Content-Transfer-Encoding:Message-Id:References:To:X-Mailer:X-ELNK-Trace:X-Originating-IP;
Received: from [] (helo=[]) by with esmtpsa (TLSv1:ECDHE-RSA-AES256-SHA:256) (Exim 4) (envelope-from <>) id 1jVRAI-000DYR-3V; Sun, 03 May 2020 22:49:06 -0400
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
From: Mitchell Erblich <>
In-Reply-To: <>
Date: Sun, 3 May 2020 19:49:04 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <>
To: Henk Smit <>
X-Mailer: Apple Mail (2.3124)
X-ELNK-Trace: 074f60c55517ea841aa676d7e74259b7b3291a7d08dfec79306390f422a66b6e8df786bfa8ceac73350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c
Archived-At: <>
Subject: Re: [Lsr] Why only a congestion-avoidance algorithm on the sender isn't enough
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Link State Routing Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 04 May 2020 02:49:10 -0000


			I think we/you are looking at two different problems: 1) a hop count of 1 or maybe two between the two end points 2) and the multiple / many hop count between the two end points.

			RFC3649 does a “ship-in-the night” type implementation of #1, where there is an excess of capacity between the two end points. It also IMO addresses the ability to burst and then close a shorter term connection.

			Thus, I think that your issue is mostly the #2 problem and the problem that most CA algorithms IMO always try to increase capacity and thus at some point must exceed capacity. TCP must find a range of capacity per flow (assuming a consistent a number of packets per sec). However, what is maybe missed (I missed it in the document) is the ability not to overshoot the TCP threshold point and trigger multiple initial congestion events in/exiting the slow-start phase.

FYI:  My implementations tend to try to identify a connection of packets-per-sec with a range using prime numbers, and possibly identify excess capacity and subdivide that capacity into future multiple connections. This type of implementation is much easier to do from a server, that implements the #1 scenario above where the server is one of the end points.

Mitchell Erblich

> On Apr 30, 2020, at 6:57 AM, Henk Smit <> wrote:
> Hello all,
> Two years ago, Gunter Van de Velde and myself published this draft:
> That started this discussion about flow/congestion control and ISIS flooding.
> My thoughts were that once we start implementing new algorithms to
> optimize ISIS flooding speed, we'll end up with our own version of TCP.
> I think most people here have a good general understanding of TCP.
> But if not, this is a good overview how TCP does it:
> What does TCP do:
> ====
> TCP does 2 things: flow control and congestion control.
> 1) Flow control is: the receiver trying to prevent itself from being
> overloaded. The receiver indicates, through the receiver-window-size
> in the TCP acks, how much data it can or wants to receive.
> 2) Congestion control is: the sender trying to prevent the links between
> sender and receiver from being overloaded. The sender makes an educated
> guess at what speed it can send.
> The part we seem to be missing:
> ====
> For the sender to make a guess at what speed it can send, it looks at
> how the transmission is behaving. Are there drops ? What is the RTT ?
> Do drop-percentage and RTT change ? Do acks come in at the same rate
> as the sender sends segments ? Are there duplicate acks ? To be able
> to do this, the sender must know what to expect. How acks behave.
> If you want an ISIS sender to make a guess at what speed it can send,
> without changing the protocol, the only thing the sender can do is look
> at the PSNPs that come back from the receiver. But the RTT of PSNPs can
> not be predicted. Because a good ISIS implementation does not immediately
> send a PSNP when it receives a LSP. 1) the receiver should jitter the PSNP,
> like it should jitter all packets. And 2) the receiver should wait a little
> to see if it can combine multiple acks into a single PSNP packet.
> In TCP, if a single segment gets lost, each new segment will cause the
> receiver to send an ack with the seqnr of the last received byte. This
> is called "duplicate acks". This triggers the sender to do
> fast-retransmission. In ISIS, this can't be be done. The information
> a sender can get from looking at incoming PSNPs is a lot less than what
> TCP can learn from incoming acks.
> The problem with sender-side congestion control:
> ====
> In ISIS, all we know is that the default retransmit-interval is 5 seconds.
> And I think most implementations use that as the default. This means that
> the receiver of an LSP has one requirement: send a PSNP within 5 seconds.
> For the rest, implementations are free to send PSNPs however and whenever
> they want. This means a sender can not really make conclusions about
> flooding speed, dropped LSPs, capacity of the receiver, etc.
> There is no ordering when flooding LSPs, or sending PSNPs. This makes
> a sender-side algorithm for ISIS a lot harder.
> When you think about it, you realize that a sender should wait the
> full 5 seconds before it can make any real conclusions about dropped LSPs.
> If a sender looks at PSNPs to determine its flooding speed, it will probably
> not be able to react without a delay of a few seconds. A sender might send
> hunderds or thousands of LSPs in those 5 seconds, which might all or
> partially be dropped, complicating matters even further.
> A sender-sider algorithm should specify how to do PSNPs.
> ====
> So imho a sender-side only algorithm can't work just like that in a
> multi-vendor environment. We must not only specify a congestion-control
> algorithm for the sender. We must also specify for the receiver a more
> specific algorithm how and when to send PSNPs. At least how to do PSNPs
> under load.
> Note that this might result in the receiver sending more (and smaller) PSNPs.
> More packets might mean more congestion (inside routers).
> Will receiver-side flow-control work ?
> ====
> I don't know if that's enough. It will certainly help.
> I think to tackle this problem, we need 3 parts:
> 1) sender-side congestion-control algorithm
> 2) more detailed algorithm on receiver when and how to send PSNPs
> 3) receiver-side flow-control mechanism
> As discussed at length, I don't know if the ISIS process on the receiving
> router can actually know if its running out of resources (buffers on
> interfaces, linecards, etc). That's implementation dependent. A receiver
> can definitely advertise a fixed value. So the sender has an upper bound
> to use when doing congestion-control. Just like TCP has both a flow-control
> window and a congestion-control window, and a sender uses both. Maybe the
> receiver can even advertise a dynamic value. Maybe now, maybe only in the
> future. An advertised upper limit seems useful to me today.
> What I didn't like about our own proposal (flooding over TCP):
> ====
> The problem I saw with flooding over TCP concerns multi-point networks (LANs).
> When flooding over a multi-point network, setting up TCP connections
> introduces serious challenges. Who are the endpoints of the TCP connections ?
> Full mesh ? Or do all ISes on a LAN create a TCP-connection to the DIS ?
> There is no backup DIS in ISIS (unlike OSPF). Things get messy quickly.
> However, the other two proposals do not solve this problem either.
> How will a sender-side congestion-avoidence algorithm determine whether
> there were drops ? There are no acks (PSNPs) on a LAN. We assume most LSPs
> that are broadcasted are received by all other ISes on the LAN. There are
> no acks. Only after the DIS has sent its periodic CSNPs, ISes can send
> PSNPs to request retransmissions. It seems impossible (or very hard) to
> me for all ISes on a LAN to keep track of dropped LSPs and adjust their
> sending speed accordingly.
> When flooding on a LAN, the receiver-side algorithm seems best. Because
> all ISes can see what the lowest advertised sending-speed is. And make
> sure they send slow enough to not overload the slowest IS. I'm not sure
> this is a good solution, but is seems easier and more realistic than
> ISIS-flooding-over-TCP or sender-side congestion-avoidance.
> My conclusion:
> ====
> Sender-side congestion-control won't work without specifying in more
> detail how and when to send PSNPs.
> Receiver-side flow-control will certainly help. I dont' know if it's
> good enough. I don't know if advertising a static value is good enough.
> But it's a start.
> I still think we'll end up re-implementing a new (and weaker) TCP.
> henk.
> _______________________________________________
> Lsr mailing list