Re: Questions about QUIC and Anycast

[Martin Thomson <[e-mail]>]

Thanks for the questions Bill,

I'll provide some short answers.  Others might be able to expand a little.

On Tue, Dec 11, 2018 at 11:18 AM William Herrin <[e-mail]> wrote:
> Question 1: Did I miss anything? Do the QUIC authors envision any
> additional ways of supporting Anycast-routed servers?

The two methods you describe are what I understand to be the primary
mechanisms.  You might find that connection ID-based routing (your
second option) to be necessary though for the reasons discussed
below...

> Question 2: Which packet contains the server preferred address?

In practice, taking some of the less-used exchanges (Retry, Version
Negotiation, a TLS HelloRetryRequest) out of the picture, the server
will often include the value in the first flight of packets it sends.
However, that can be a couple of round trips away from the end of the
handshake, because the handshake is multiple packets and might need to
be repaired due to packet loss.

> Can the client start
> trying to use the preferred address with its second packet or does it
> have to wait?

The client has to wait, because - until the handshake is complete - it
can't authenticated the preferred address value provided by the
server.  Also, validation of the path to that address likely takes
time.  That basically forces the server to use connection IDs to
ensure continuity.  A server might rely on the route being stable for
the duration of the handshake, though that would rely more on having
timely termination of the connection; keeping in mind that our best
mechanism, stateless reset, probably isn't viable at this point.

> Is it allowed to continue using the non-preferred
> address and, if so, will it fail over to attempting the preferred
> address if its communication fails in some way (no response or
> rejection from the server at the non-preferred address)?

Yes, a client can choose to ignore the preferred address.  And yes, I
believe that the working group understands the implications of this.
If someone didn't and finds this distressing, yell.

> Question 3: What picks the connection ID? Does the API allow the
> application software to influence the selection of connection ID in
> any way?

The server picks connection IDs that the client puts in packets.  That
selection really has to be done in consultation with load balancers or
the folks managing routing configurations.

There's work on a protocol for managing this:
https://tools.ietf.org/html/draft-duke-quic-load-balancers-03

> Question 4: Is the connection ID inside or outside the encrypted
> portion of the packet?

The value is unencrypted, but it is authenticated.  So it can be read,
but not changed.

> Question 5: Fragmentation needed / Packet Too Big messages originate
> from an intermediate router that often has.a different return path
> than the route taken by packets from the endpoint. How is QUIC
> intended to deal with packet too big messages which arrive at a
> different node in the Anycast cluster than the one which sent the
> overlarge packet?

The details of how to deal with PTB messages is in
https://quicwg.org/base-drafts/draft-ietf-quic-transport.html#packet-size

We realize that there is a limitation here: the PTB message won't
contain the connection ID selected by the server, so it can get
misrouted.  Kazuho suggested using a dummy packet with a long header
on PMTUD probes for that reason (those packets can be skipped and
contain both connection IDs), but I don't think that we've agreed to
formalize that technique, so there might be some privacy implications
we haven't considered.