Re: Packet number spaces in multipath (was Re: What to do about multipath in QUIC)

[Jana Iyengar <[e-mail]>]

Mirja,

I'm referring to what Christian was summarizing below. Separate PN spaces
but path ID is implicit as the sequence number of the connection ID, and
ACKs reflect this sequence number.

- jana

On Wed, Dec 2, 2020 at 3:02 AM Mirja Kuehlewind <
[e-mail]> wrote:

> Hi Jana,
>
>
>
> can you maybe confirm what you mean by “the design” below just to make
> sure we are all on the same page: Is that different PN spaces per path, but
> using the same key on all paths with CIDs as part of the nonce?
>
>
>
> Thanks!
>
> Mirja
>
>
>
>
>
> *From: *QUIC <[e-mail]> on behalf of Jana Iyengar <
> [e-mail]>
> *Date: *Wednesday, 25. November 2020 at 04:35
> *To: *Christian Huitema <[e-mail]>
> *Cc: *IETF QUIC WG <[e-mail]>, Kazuho Oku <[e-mail]>
> *Subject: *Packet number spaces in multipath (was Re: What to do about
> multipath in QUIC)
>
>
>
> (I'm taking Spencer's suggestion to spin off a new thread.)
>
>
>
> Christian, Kazuho,
>
>
>
> Slowly catching up on this, and apologies if I'm missing anything that was
> previously discussed in the centi-thread earlier.
>
>
>
> If I understand the design correctly, it makes sense to me, and is very
> close to what we had implemented in Chromium a while ago.
>
>
>
> Having thought about this problem several times in the past, I'd like to
> share a few points that come to mind.
>
>
>
> First though, a point on terminology: the receiver maintains a separate
> "ReceivedPackets" for each CID, probably for each CID sequence number
> (CSN). Let's please not call this a SACK Dashboard, to avoid confusion.
>
>
>
> On the question of sending more than 2^32 packets, I think that resetting
> the packet number (PN) is ok on new CIDs. I don't see why a sender would
> need to maintain continuity across multiple paths anyways, since the CC and
> loss recovery contexts are going to be different across paths. A sender
> _could_ still maintain these packets in the same "SentPackets" structure if
> it wants to, it would need an internal representation of CSN+PN to key off.
>
>
>
> ACK Frames:
>
> ------------------
>
> Kazuho pointed out that when acknowledging, the ACK frame format should
> include CSN. I agree. I would argue for a design where a receiver uses an
> ACK frame per CSN (and encodes the CSN explicitly). There are multiple
> values for doing this, the primary being that you benefit from compression
> when PNs are contiguous within a CSN.
>
>
>
> Return Path:
>
> -----------------
>
> There are other ways to decide which return path to send an ACK on this,
> but I would propose that a receiver respond on the most recently active
> forward path. That is, the path on which a packet was most recently
> received. This has the natural effect that a sender that wants to
> distribute traffic in a particular way also causes ACKs to be distributed
> similarly across the corresponding reverse paths.
>
>
>
> RTT measurements:
>
> ---------------------------
>
> The return path for ACK frames will impact RTT measurements. That is fine.
> It is more important that information reach the sender as soon as possible
> than that it should not affect RTT measurements; we can fix the sender
> to measure and compensate as necessary. The estimated RTT statistics
> reflect the distribution of samples, and if both paths are being used, then
> the SmoothedRTT will reflect the expected value based on the traffic
> distribution across paths.
>
>
>
> That said, it might be useful to track some new stats, especially about
> how much later is a "late ack" -- an ACK frame that contains no useful
> information -- is received. I'd have to think a bit more about this, but I
> think we can devise a stat here. This gives us useful information on the
> longest return path, which we can then explicitly use as part of the PTO
> computations, to compensate for the fact that the RTT is based on the
> shortest return path. (I would _not_ use this stat in the time-based loss
> detection timer,  but PTO ought to be fine.)
>
>
>
> - jana
>
>
>
> On Tue, Nov 17, 2020 at 9:42 AM Christian Huitema <[e-mail]>
> wrote:
>
> I have been thinking about variations of that. I think we are making
> progress here.
>
> If we follow your design, we get two constraints:
>
> 1) That the receive maintains an acknowledgement list based on the CID
> through which the packets are received.
>
> 2) That the senders guarantee that the same sequence number will not be
> used more than once with a specific CID.
>
> The main implementation cost is for receivers. They have to allocate and
> maintain a "SACK Dashboard" in the context of each CID that they issue.
>
> Senders have lots of control. For example, the "only once" condition is
> also met if a simple sender uses a single number space, as long as it does
> not send more than 2^32 packets. That makes the design reasonably easy to
> implement for constrained implementations.
>
> Zero length CID are still possible, but that means the receiver supports
> only one PN space per sender. Multipath is not impossible, but you end up
> managing a single RTT and a single recovery structure. Not very good, but
> similar to what happens if multipath is implemented at the IP level.
>
> There is still an issue for coordinating the take down of a path. Suppose
> that a client was using both Wi-Fi and LTE, and moves out of Wi-Fi range.
> The server will find out eventually that the packets sent to the Wi-Fi path
> are never acknowledged, but that may take some time. It would be better if
> the client could send a message saying something like "Abandon this path".
> That's not the same semantic as "retire this CID". We need a new frame for
> that.
>
> "Abandon this path" is an extreme case. There are half-way steps, like
> manage the relative priority of a path.
>
>