Re: [Plus] Supporting connection tracking and basic diagnostics: a minimal PLUS

Brian Trammell <> Mon, 12 December 2016 10:08 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id A01B91295A8 for <>; Mon, 12 Dec 2016 02:08:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.798
X-Spam-Status: No, score=-4.798 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RP_MATCHES_RCVD=-2.896, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Ga2OvZUo5YmX for <>; Mon, 12 Dec 2016 02:08:02 -0800 (PST)
Received: from ( []) by (Postfix) with ESMTP id 380C1129587 for <>; Mon, 12 Dec 2016 02:08:01 -0800 (PST)
Received: from [IPv6:2001:470:26:9c2::7ea] (unknown [IPv6:2001:470:26:9c2::7ea]) by (Postfix) with ESMTPSA id 8D90B1A1078; Mon, 12 Dec 2016 11:07:30 +0100 (CET)
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Content-Type: multipart/signed; boundary="Apple-Mail=_994CE9AE-B9EE-4A35-A2F3-3249FA425291"; protocol="application/pgp-signature"; micalg=pgp-sha512
X-Pgp-Agent: GPGMail
From: Brian Trammell <>
In-Reply-To: <02ec01d25378$83799550$8a6cbff0$>
Date: Mon, 12 Dec 2016 11:07:29 +0100
Message-Id: <>
References: <> <> <> <> <02ec01d25378$83799550$8a6cbff0$>
To: Christian Huitema <>
X-Mailer: Apple Mail (2.3124)
Archived-At: <>
Cc:, Ian Swett <>, Tommy Pauly <>
Subject: Re: [Plus] Supporting connection tracking and basic diagnostics: a minimal PLUS
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: "Discussion of a Path Layer UDP Substrate \(PLUS\) protocol for in-band management of in-network state for UDP-encapsulated transport protocols." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 12 Dec 2016 10:08:04 -0000

hi Christian, all,

> On 11 Dec 2016, at 07:33, Christian Huitema <> wrote:
> On Saturday, December 10, 2016 4:35 PM, Tommy Pauly wrote:
>> I agree with Ian that the first option is probably easier and more viable as a
>> solution. The second option is attractive in how it makes the association and
>> confirmation values less predictable, and thus more trustworthy. However,
>> the extra state required to debug and reconstruct the flow does seem too high
>> if want wide adoption.
> I too think that the first option is more practical.

Having had a weekend to think a bit more about both of them, I'll add my voice to this chorus.

> I am a bit concerned that the "token" suggested in the second option does not have any link to the mechanism of the transport protocol. That means lazy implementation could take their sweet time and "rotate the token" once in a blue moon, making it seriously unfit for RTT assessment.

There are other ways in which the RTT assessment can go awry. You need a little more state (and, on path, an extra addition) to do RTT measurement with the first option, but it has the advantage over its TCP counterpart that the packet number *always* increments, even for ACKs, and the echo number is always the max packet number seen, so you can get good upstream RTT component measurements even if all the client sends is ACKs.

> In the first option, the echo is the "last seen sequence number" that would otherwise be carried in the ACK frame. If we move it out of the ACK frame and into the header, then we have pretty much a guarantee that applications will do it right. I like that.

Well, to be fair, the first option also allows two endpoints running a private transport protocol to collude to lie to the path about their RTT by agreeing to manipulate echo numbers. I'm not sure it's either possible or useful to prevent that, though.

> I also like that the scheme does not make the "connection start" special, and works just as well after a rerouting or a NAT state change.

That was a hard requirement of our design exercise. It's also the main problem I can see with the revealed-hash stop signal: you need to send the proof value somewhen, and the exposure of that proof is a special "connection start", unless it is exposed periodically (which comes with a cost in overhead).

> OTOH, there is going to be a "feature interaction" between this sequence+echo scheme and the senders' strategy of skipping some sequence numbers in order to verify that the recipients are not making up ACKS. The intermediate boxes will have no way to distinguish between a packet number voluntarily skipped by the sender, and a packet loss between the sender and the middlebox. I hope this won't affect the "wireshark diagnostics" too much.
>> When we are looking at the first option, you mentioned strategies to avoid on-path
>> injection attacks of stops.
> I like the "two ways stop". It definitely increases the robustness. Also, sending the sequence number and last seen in "clear text" solves the "guess the sequence number" issue for the "rebooted server." Maybe we can use EKR's scheme after all.

The two-way stop and the revealed-hash stop are independent, so you could also do both if you wanted to.

>> ... Is there any attack that could be made regarding the PSN? If the attacker can modify
>> PSN values or inject packets with higher PSN values in either direction, what side effects
>> can we see? I'm assuming that the endpoints themselves will be able to validate the
>> authenticity of the fields, but could a middle box that is trying to monitor the
>> plausible-ness of the PSN get thrown off? Or does it only ever adjust its plausibility
>> window when it sees a PSN acknowledged? (To that end, if an acknowledgment
>> value is faked, what does that do?)
> That's a very good observation, and in fact it applies to current QUIC as well. A man-on-the side can inject a series of plausible packets before the sender does. The "real" packets coming from the sender will repeat packet numbers that are 'already seen'. A middlebox that is too clever might drop those in an attempt to "remove duplicates." We can always say that middleboxes should not try to be too clever, but that's a bit like saying that the bubonic plague should be nicer...

This might be an argument for recommending against plausibility checks for middleboxes for anything other than the two-way stop. Unlike with TCP, in any case they cannot assume anything about window dynamics based upon assumed knowledge of the reliability or congestion control schemes in place. Whether this recommendation would be widely followed is a different question.

Since the endpoints should be able to detect a middlebox-injected packet (at least as long as there's a requirement that *every* packet have an (encrypted) transport header above the common unencrypted header, which, IMO, there is), really these attacks are all about middleboxes messing with each other. If the endpoint can always detect, then it can also close and attempt to reinitiate, or at least throw up a "your network is messing with you" error.