Re: [Masque] MASQUE extensibility

[David Schinazi <[e-mail]>]

Thanks! Yes, that's exactly what I meant by "repeatable" - perhaps
"repeated" would have been a better choice of word here.

David

On Wed, Jan 6, 2021 at 3:26 PM Christopher Wood <[e-mail]> wrote:

> On Wed, Jan 6, 2021, at 1:06 AM, David Schinazi wrote:
> > Hi Chris,
> >
> > Thanks for your note, I think that we should strive to reach consensus
> > on extensibility goals, requirements, and means around our upcoming
> > interim. However, I don't understand all of your individual questions
> > (for example, I don't understand what the word "fixed" means in the
> > first question). Perhaps allow me to try to phrase these questions
> > differently, to attempt to distill fundamental concepts.
> >
> > What we're mainly building is a way to convey data (e.g. the payload of
> > a UDP packet) and metadata (e.g. the UDP port number); I'll further
> > subdivide metadata into two categories: repeatable-metadata (e.g. the
> > IP address of the target-server, ECN-ECT(0) vs CE) and
> > non-repeatable-metadata (e.g. the timestamp of when the UDP packet was
> > received). One way we could have built CONNECT-UDP was to send all
> > repeatable-metadata and non-repeatable-metadata in every single packet,
> > i.e. every single DATAGRAM frame carries the target-server IP and port.
> > But, in order to improve efficiency and simplify management on the
> > proxy, we decided to associate repeatable-metadata with a flow ID, and
> > then only transmit the flow ID in DATAGRAM frames. This introduced the
> > need for managing lifetimes (how long does the proxy maintain the
> > mapping between flow ID and target-server IP+port?). We decided to tie
> > this lifetime to the existence of the bidirectional stream that carried
> > the CONNECT-UDP request. We did this because it resembled how CONNECT
> > worked, but also because it was efficient and relatively simple to
> > implement.
>
> Am I correct in assuming that "repeatable" here means "possibly sent more
> than once across different packets"? I assume so, given that the target
> address is one such example.
>
> Either way, this is a *much better* phrasing of the fundamental problem.
> Thanks for writing it up. :-)
>
> > Now, to dive into your questions, let's focus on the example of
> > CONNECT-UDP with ECN support. ECN adds two more bits of metadata per
> > packet, and those bits are repeatable-metadata. From my point of view,
> > the most logical way to encode those is to reuse the mechanism we built
> > for vanilla CONNECT-UDP: we associate this repeatable-metadata with a
> > flow ID. The latest version of the drafts (see links below) does this
> > by having the CONNECT-UDP request/response negotiate four flow IDs. But
> > we could have built this differently.
> >
> > Question 1: should repeatable-metadata be sent in each DATAGRAM frame,
> > or associated with something like a flow ID?
> >     The efficiency benefits lead me to conclude that associated is
> > preferable.
> >
> > Question 2: should the lifetimes of these mapping be distinct or tied?
> > I.e., should we be able to get in a state where we have a mapping for
> > ECN-CE but not for ECN-ECT(0)?
> >     Separate lifetimes can lead to odd bugs, so I'd prefer to tie them.
> > This means not using four bidirectional streams per CONNECT-UDP with
> > ECN flow.
> >
> > Question 3: Where do we encode this mapping? Flow ID or new QUIC or H3
> > frames?
> >     The flow ID is a simple concept that is simple to implement,
> > especially if you don't control the underlying QUIC stack, so I prefer
> > those.
> >
> > With this reasoning, I think that the current proposal is a good
> > solution to our constraints. But if someone disagrees with these
> > conclusions, please let me know - perhaps there are other concerns that
> > I'm not aware of.
> >
> > draft-schinazi-masque-h3-datagram-04
> > <https://tools.ietf.org/html/draft-schinazi-masque-h3-datagram-04>
> > draft-ietf-masque-connect-udp-03
> > <https://tools.ietf.org/html/draft-ietf-masque-connect-udp-03>
> > draft-schinazi-masque-connect-udp-ecn-01
> > <https://tools.ietf.org/html/draft-schinazi-masque-connect-udp-ecn-01>
> >
> > Cheers,
> > David
> >
> > On Tue, Jan 5, 2021 at 9:05 PM Christopher Wood <[e-mail]>
> wrote:
> > > Happy New Year, all!
> > >
> > > Thanks to everyone who chimed in on this thread! To summarize
> responses we received so far, it seems clear that we want extensibility for
> CONNECT-UDP and the future IP proxying mechanism, and that these should
> exist in separate documents. David’s draft specifying an ECN extension [1]
> is one such example. It also seems clear that HTTP headers make for a
> perfectly fine extension point for CONNECT-X requests.
> > >
> > > What remains unclear is how to support extensions within a proxy
> context, i.e., a CONNECT-UDP stream. Martin nicely outlined some of the
> variants, such as encoding things in per-datagram framing bits, negotiating
> and encoding information in flow IDs, or using different frame types
> entirely. We should try to converge on which of these options we need for
> generic tunneling or proxying use cases.
> > >
> > > To that end, here’s a couple questions that might help us get there:
> > >
> > > - Should flow ID interpretation be negotiated as part of an extension,
> or should it be fixed for CONNECT-UDP? (The latter is the approach taken in
> [1].)
> > > - What are the tradeoffs between per-datagram signalling state versus,
> say, a different datagram frame type for different use cases? And which of
> these is maximally useful for CONNECT-UDP use cases?
> > > - What are the tradeoffs between per-datagram and flow ID signalling?
> > >
> > > We hope these (or related questions) can be discussed before and
> during the interim.
> > >
> > > Thanks,
> > > Chris and Eric
> > >
> > > [1]
> https://tools.ietf.org/html/draft-schinazi-masque-connect-udp-ecn-00
> > >
> > > On Wed, Dec 9, 2020, at 12:03 PM, Martin Duke wrote:
> > > > Thanks for starting this discussion, Chris.
> > > >
> > > > With my AD hat on, I would personally interpret the charter to allow
> > > > CONNECT-UDP to provide all the functionality one might have in a UDP
> > > > socket, except those (like multicast) explicitly forbidden. This
> > > > includes ECN, DSCP, etc, although the WG is welcome to decide that
> some
> > > > or all of these functions aren't important. As AD, I don't have a
> > > > strong opinion today as to whether these end up as one draft or
> > > > multiple drafts. Certainly, it makes sense to spell out the design
> in a
> > > > separate draft first and later decide whether or not to integrate it
> > > > into the base design, as quicwg did with the spin bit and ECN
> support.
> > > >
> > > > If the CONNECT-UDP method is designed so as to be literally not
> > > > extensible to support these use cases, I would consider that a
> > > > significant flaw. I don't think that's the case.
> > > >
> > > > Speaking as an individual, the hardest extensibility problem would
> seem
> > > > to be per-datagram information. There are four different resources
> we
> > > > could use for this:
> > > > - an additional QUIC frame type codepoint, with the second type
> having
> > > > additional per-datagram information
> > > > - an additional H3 frame type codepoint, with the second type having
> > > > additional per-datagram information
> > > > - flow ID codepoints, as David suggests. If we want to support DSCP,
> > > > this means we need 8 bits per flow just for this (but then, we have
> > > > 2^62 of them!)
> > > > - Make every H3 datagram frame encode this information -- if we were
> to
> > > > choose this option, we will probably end up using more H3 frame
> types
> > > > as Webtransport, etc. will probably not want the overhead.
> > > >
> > > > To me, the first two are the least profligate, and the second option
> > > > (another H3 frame) is the right layer to do it. But none of these
> > > > resources are scarce and this is mainly an aesthetic preference.
> > > >
> > > > Broadening from the per-datagram issue, we are likely filling up
> some
> > > > sort of HTTP or H3 registry with whatever extension points we
> specify,
> > > > so that community's opinion is valuable here.
> > > >
> > > > Martin
> > > >
> > > > On Wed, Dec 9, 2020 at 10:01 AM Mirja Kuehlewind
> > > > <mirja.kuehlewind=[e-mail]> wrote:
> > > > > 1) Yes, every new protocol should be extensible. Lucas and others
> mentioned that H3 provides already some extensibility, however, as soon as
> you have send the CONNECT there is not much H3 "left" and therefore we
> should make sure that we consider options for additional extension
> mechanisms later in the life time of a CONNECT request.
> > > > >
> > > > > 2) People did mention in-stream (or I would say
> in-forwarding-association - I  think we need to agree on terminology here
> to make sure we talk about the same thing) control data. So I think there
> is information that you want to signal that is associated to a forwarding
> stream or even a certain packet, where it probably make sense to send this
> data within the respective forwarding stream, however, there might be other
> cases where you want to signal something based on some event in the proxy,
> including negotiation when or before the CONNECT is sent, or general
> information about proxy state or capabilities which maybe be independent of
> any active forwarding association. We also need to consider appropriate
> ways to signal such information and I think all (new) signaling we define
> should be extensible.
> > > > >
> > > > > 3) I think the extension points itself need to be described in the
> base spec. I also still think that ECN handling should be part of the base
> spec as this is an important functionality that is in my view not optional
> if we ever want to see it used. However we can of course start with
> separate documents and merge in when ready. Still, we first must agree
> about the kind of extension points we want to use.
> > > > >
> > > > > Mirja
> > > > >
> > > > >
> > > > > On 04.12.20, 21:19, "Masque on behalf of Christopher Wood" <
> [e-mail] on behalf of [e-mail]> wrote:
> > > > >
> > > > >     One point raised during our IETF 109 meeting was
> extensibility. A number of different extensions have already been
> discussed, including: IP header compression, ECN signals, and QUIC-aware
> proxy support. It seems clear that the MASQUE use-cases require some amount
> of extensibility in the core mechanism. What we’d like to determine is how
> much extensibility is needed.
> > > > >
> > > > >     Given that our charter is somewhat vague on what extensions
> are in scope [1], there seem to be (at least) three questions we should
> answer if we are to embark on this extension work:
> > > > >
> > > > >     1) Do we want CONNECT-UDP and a future mechanism for IP
> proxying to be extensible?
> > > > >     2) If yes, which parts of each protocol need extensibility,
> and to what extent?
> > > > >     3) If yes, do we want to adopt these extensions as distinct
> documents? (An ECN signal, for example, might be included as an extension
> in the CONNECT-UDP document, or it might be part of a separate document.
> Conversely, extensions for QUIC-aware proxying seem likely to be a separate
> document.)
> > > > >
> > > > >     We'd like to hear answers to these three questions from the
> WG. Converging here will help us better determine what is in scope going
> forward. To that end, please share your thoughts on these questions before
> December 18. We'll assess and see where we are at that time.
> > > > >
> > > > >     Thanks,
> > > > >     Chris and Eric
> > > > >
> > > > >     [1] Relevant text in the charter (
> https://datatracker.ietf.org/doc/charter-ietf-masque/) reads as follows:
> > > > >
> > > > >     The primary goal of this working group is to develop
> mechanism(s) that allow configuring and concurrently running multiple
> proxied stream- and datagram-based flows inside an HTTPS connection. These
> mechanism(s) are collectively called MASQUE. ***The group will specify HTTP
> and/or HTTP/3 extensions to enable this functionality.***
> > > > >
> > > > >     (emphasis ours)
> > > > >
> > > > >     --
> > > > >     Masque mailing list
> > > > >     [e-mail]
> > > > >     https://www.ietf.org/mailman/listinfo/masque
> > > > >
> > > > > --
> > > > > Masque mailing list
> > > > > [e-mail]
> > > > > https://www.ietf.org/mailman/listinfo/masque
> > >
> > > --
> > > Masque mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/masque
>