Re: [Masque] QUIC non-encapsulation and CIDs

[Tommy Pauly <[e-mail]>]

Hi Martin,

Thanks for the read-through! Comments inline

> On Nov 9, 2022, at 11:48 AM, Martin Thomson <[e-mail]> wrote:
> 
> I've been looking more closely at draft-pauly-masque-quic-proxy.  I think that we can fix the protocol so that the use of this mechanism isn't a strict privacy reduction for the general case.  But I want to start with the connection ID design, which I find very confusing in its current form.

Apologies for the confusion. Likely moving diagrams and examples earlier on would help.

> 
> I wasn't able to tell exactly how the connection ID mappings work in the draft.  The structure of the document is a little confusing.  I really hoped that it would discuss how things work rather than just document state and capsule formats.
> 
> The idea something like this:
> 
> 1. Client -> Server direction
> 
> The client tells the proxy about each connection ID used by the server.  The proxy tells the client which connection ID to use for that connection ID when sending packets to the proxy.
> 
> For this the client needs to tell the proxy:
> a) the connection ID the server has provided
> b) (maybe) the stateless reset token that the server will recognize if we want the proxy to be able to terminate the flow
> 
> I want to put a pin in the stateless reset design here, because there are a lot of requirements we might need to discuss regarding that.  More in a follow-up, I think.  Stateless resets that aren't visible to the proxy are not great, but we might want to avoid having the proxy be able to terminate the flow at the client.  There is a simple design for this too - maybe - but I need to think about that some more.
> 
> The proxy can then tell the client
> a) the (virtual) connection ID that the proxy will map to the provided server connection ID
> 
> Here, the client is responsible for forwarding connection IDs from the server to the proxy so that it can be assigned a virtual connection ID.

I believe the document is doing exactly what you’re describing here for this direction.

Taking the example in the document, the client tells the proxy that the CID that the server has provided. (It also includes an optional stateless reset token for the target server, but I realize now that the example didn’t get updated for that.)

The proxy then tells the client a virtual CID that the proxy will map to the target server CID. It also here can include an optional stateless reset token that it will send in response to the client using the virtual CID.

https://tfpauly.github.io/quic-proxy/draft-pauly-masque-quic-proxy.html#name-example

STREAM(44): DATA                -------->
  Capsule Type = REGISTER_TARGET_CID
  Connection ID = 0x61626364
  Stateless Reset Token = Token

           <--------  STREAM(44): DATA
                        Capsule Type = ACK_TARGET_CID
                        Connection ID = 0x61626364
                        Virtual Target Connection ID = 0x123412341234
                        Stateless Reset Token = Token


> 
> Note that I think that virtual connection ID is not a great name, this is a mapped connection ID or a replacement connection ID.  Maybe an overlaid connection ID.

Sure, that’s a fine bike shed. I like “mapped”.
> 
> 
> 2. Server -> Client direction
> 
> The client will need to provide the server with connection IDs that the proxy will accept, so...
> 
> The client tells the proxy:
> a) the connection ID it wants to use
> 
> The proxy responds with:
> a) the connection ID that it wants the client to pass to the server
> 
> Basically, the client is responsible for putting connection IDs from the proxy into the connection on this side.

I don’t quite follow this case.

The proxy will be receiving packets from the server to the client on a socket that the proxy opens up to the server. Since this is a socket with a local port and address that the individual server can control, I don’t think it would normally need to use a CID that it selects for load balancing purposes.

The other reason to let the client choose its own CID that it represents to the server is that this allows the client to initiate an Initial packet in the first flight before it gets a response from the proxy.

Looking at the example again, the client tells the proxy about the CID it is sending to the client, and it also tells the proxy about the “virtual/mapped” CID that it wants the proxy to use on the leg back to the client.

STREAM(44): DATA                -------->
  Capsule Type = REGISTER_CLIENT_CID
  Connection ID = 0x31323334
  Virtual CID = 0x62646668
  Stateless Reset Token = Token

           <--------  STREAM(44): DATA
                        Capsule Type = ACK_CLIENT_CID
                        Connection ID = 0x31323334

If you see a need for the proxy to choose the actual CID, can you explain it in more detail?
 
> 
> 
> All of these are request-response exchanges, because the mapping will need to be clear.  Also, that same identifier can be used to remove connection IDs once the client or server retire them.  For that, I think that using the QUIC sequence number is fine as it might allow a bulk removal at the proxy if Retire Prior To is used.

Right, the REGISTER/ACK exchanges are all request and response pairs that are identified by a specific connection ID. If we want to add the extra sequence number in here, that could be fine too.
> 
> Things I didn't solve: connection ID limits (the proxy will want some, how does the client convey this?),

Do you mean the number of connect IDs the proxy is willing to understand and process? It can reject if a limit is exceeded, I think.

> removal messages (easy),

Indeed, the CLOSE capsules do this.

> and connection ID length constraints and the effect of different lengths on MTU (this should be minor, but it will need to be mapped out).

There is some text on the different lengths, but it can likely be expanded.
> 
> With this design, the proxy could move all forwarding logic to a load balancer and never touch packets again.  The load balancer only needs to support QUIC-LB for that to work (with encrypted connection IDs, of course).

The forwarding logic right now is something we put into eBPF on the same box, but if you have a way to program forwarding rules onto a different box that can also share the sockets that the proxies establish to the target servers, I could see this working. 

>  Of course, this will need to be revisited in my follow-up as the workload increases somewhat.

Is your follow-up a proposal on how to add an extra encryption layer, or something else?

Thanks,
Tommy
> 
> Cheers,
> Martin
> 
> -- 
> Masque mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/masque