Re: [Ohai] Proxying WebSockets and gRPC

On Tue, Sep 26, 2023 at 3:01 PM Christopher Wood <caw@heapingbits.net>
wrote:

>
> On Sep 26, 2023, at 2:54 PM, Eric Orth <ericorth@google.com> wrote:
>
> Maybe it's my unfamiliarity with gRPC, but why wouldn't that simple
> REST-like API fit over "normal" OHTTP without the need for chunked OHTTP or
> additional extensive guidance? What is the gRPC layer adding that is both
> helpful for the interaction and that you can't just send as a simple
> request and response?
>
>
> gRPC supports streams of messages, which is why I brought chunked OHTTP
> into the picture. If you don’t care about streams, then yes, you should in
> theory just be able to use OHTTP out of the box.
>
>
> For the most part, I think we've avoided the need for guidance on adapting
> specific protocols to OHTTP because it's either obvious or it's not well
> suited to OHTTP.  If something like gRPC adds a bunch of stuff that doesn't
> obviously fit into OHTTP, then the answer is probably to not use gRPC and
> just extract out the requests and responses into a more pure OHTTP
> interaction.  Just like how OHTTP is not actually compatible with generic
> HTTP and you can't expect any arbitrary HTTP interaction to cleanly go over
> OHTTP without massaging it into OHTTP compatibility, we shouldn't expect
> other HTTP-like protocols to go cleanly over OHTTP either.
>
>
> Anecdotally, what constitutes “a bunch of stuff that doesn’t obviously fit
> into OHTTP” is not obvious to everyone, which is partly why I raise this
> topic. I appreciate that they may be obvious to folks on this list, but I
> don’t think this list represents the audience of people who may want to
> actually use these technologies in practice.
>

Fair enough, but if it's not obvious to folks on this list how gRPC could
obviously fit into OHTTP, then the answer for best guidance to people off
the list is probably something along the lines of "Don't use gRPC over
OHTTP.  If you have a simple streamed-message API without needing
persistence or sessions, extract out your streamed messages and send them
directly over chunked OHTTP."

>
> Best,
> Chris
>
>
> On Tue, Sep 26, 2023 at 10:26 AM Christopher Wood <caw@heapingbits.net>
> wrote:
>
>> I agree with the technical points below around privacy, security, and
>> cost, but I feel like the core question was missed. To help sharpen it,
>> let’s ignore WebSockets for the time being and focus on gRPC.
>>
>> First, I think we all understand that OHTTP targets a constrained set of
>> HTTP use cases — we debated about the name for a while because of this very
>> reason! That constraints are basically what Tommy described: there is no
>> state across requests and there is no notion of a session. My claim is that
>> gRPC can be used to build application APIs with these exact same semantics.
>> Moreover, given that gRPC uses HTTP under the hood, it seems perfectly
>> reasonable to use OHTTP to provide the desired privacy properties under
>> these constraints.
>>
>> To provide an example, imagine you were building a weather app. The app
>> has one API — give me the forecast for a particular location (where
>> location could be city, lat/lon coordinates, or whatever). We could build
>> this API in a REST-like way by defining a weather resource, and then
>> GETting the weather for a particular location, e.g.,
>>
>>    Client->Server: GET /weather?location=newyorkcity
>>    Server->Client: 200 OK <weather>
>>
>> We could also build this API in an RPC-like way by defining a procedure
>> that computes the weather based on a location message and returns the
>> result. Handwaving over the gRPC layer, interacting with this API using
>> HTTP is then something like this:
>>
>>    Client->Server: POST /weather <body includes protobuf of location
>> message>
>>    Server->Client: 200 OK <body includes protobuf of weather forecast
>> message>
>>
>> These aren't the world’s greatest APIs, but hopefully they get the point
>> across. Specifically, in both cases, requests to the app are stateless and,
>> ideally, unlinkable. The app should not learn anything about the client’s
>> identity (IP address) when servicing a request. There is no notion of a
>> session.
>>
>> One might reasonably use OHTTP to mask the client’s IP when interacting
>> with the REST variant. It’s trivial to do and quite cheap. It’s something
>> that has been deployed for a variety of similar use cases today, ranging
>> from DNS to application-specific APIs [1]. I think it’s reasonable to also
>> use OHTTP when interacting with the RPC API. Or, rather, I see no
>> convincing technical reason why you would want to apply MASQUE here instead
>> of OHTTP.
>>
>> With this as motivating context, my open-ended question was basically how
>> folks think about adding proxy support for these different types of
>> application APIs. Taking a step further back, I think it might be useful if
>> this community produced some sort of guidance on how to design APIs and
>> choose the appropriate proxy technology for developers who write such apps,
>> and for developers who maintain legacy apps. The folks on this list have no
>> doubt thought about and had conversations with people about when and why we
>> might use OHTTP vs MASQUE and how they fit into the application
>> architecture, so perhaps we ought to try and write down some of those
>> considerations for the benefit of those outside of this list.
>>
>> Thoughts?
>>
>> Best,
>> Chris
>>
>> [1] https://flo.health/press-center/anonymous-mode-feature
>>
>> On Sep 25, 2023, at 11:29 PM, Tommy Pauly <tpauly=
>> 40apple.com@dmarc.ietf.org> wrote:
>>
>> I agree with Martin that for something that has a generic stream like
>> WebSocket or anything with multiple requests, you’re going to be more
>> stateful, and you’re better off using a MASQUE-flavored proxy.
>>
>> OHTTP is suitable for request-response pairs that benefit from being
>> highly unlinkable, but still aren’t “sessions” where there is
>> back-and-forth state.
>>
>> For cases like the chunked OHTTP variant, we shouldn’t use that as a way
>> to shoehorn in arbitrary streams, but to better accommodate cases where one
>> of the messages is better processed in pieces. Supporting 1XX responses in
>> one case, and the other (which I’m interested in) is when you have
>> responses that may be short or very long, and in the long cases may be slow
>> to generate and process.
>>
>> Thanks,
>> Tommy
>>
>> On Sep 25, 2023, at 7:36 PM, Martin Thomson <mt@lowentropy.net> wrote:
>>
>> (Duplicate?)
>>
>> Either way, I don't know if something like gRPC or thewebsocketprotocol
>> makes sense to use here.  If you want crazy, then why not also allow the
>> modern HTTP CONNECT variants?  Yes, you could, but the "should" question is
>> probably a more important one.
>>
>> Reasons that you might not want to use OHTTP for either of these:
>>
>> 1. Replay risk.  OHTTP doesn't protect against replay attacks by a relay
>> (or, if you are crazy enough to send something TLS early data toward the
>> relay, arbitrary network attackers).  It is hard to reason about the replay
>> risk with arbitrary protocols like these.  Maybe we could layer liveness
>> checking on top, but is that really the direction we want to go in?
>>
>> 2. Latency savings are marginal when costs can be amortized.  OHTTP does
>> have lower overhead than something like MASQUE, but once the overheads in
>> MASQUE are amortized over a longer-lived session, is this really that
>> valuable?
>>
>> 3. Cost is cost and I don't see how a MASQUE proxy is materially
>> different than OHTTP in this case.  Maybe the proxy spends more time on
>> crypto, but the trade-off is the replay attack risk.
>>
>> gRPC is a little more complex than a simple one-in-one-out query, which
>> suggests further issues with OHTTP on top of those.  Multiple requests mean
>> that overheads build up and MASQUE starts looking even better for that.
>>
>> Do you have something specific in mind?
>>
>> On Tue, Sep 26, 2023, at 04:31, Christopher Wood wrote:
>>
>> Hi folks,
>>
>> Thanks to the contributions of this group and the companion MASQUE
>> group, we now have a helpful suite of protocols for proxying
>> application traffic in a variety of scenarios. MASQUE gives us a
>> generic way to proxy connections to existing servers, and OHTTP gives
>> us an application-specific way to proxy transactional messages to
>> supporting applications. These two cover quite a lot of ground as
>> currently specified, allowing us to proxy browser traffic as well as
>> most HTTP-based API traffic.
>>
>> In some of my discussions about "proxying all the things," two use
>> cases have emerged as somewhat common: proxying traffic for APIs built
>> on WebSocket and gRPC. This is unsurprising given the popularity of
>> these two technologies. It seems prudent that we have a reasonable
>> answer for those who want to support these use cases.
>>
>> One straightforward answer is to just use MASQUE! This is a perfectly
>> fine approach if you have a proxy to use. However, perhaps MASQUE is a
>> bit too expensive -- for some definition of expensive -- for some
>> deployments, and OHTTP is a more preferred option.
>>
>> Given that we now have chunked OHTTP as a proposal [1], I wonder what
>> folks think about using this variant to support WebSockets and gRPC.
>> gRPC supports transactional and streaming requests and responses, which
>> fits well with normal and chunked OHTTP, and if you squint at
>> WebSockets you might view it as a long-standing request and response
>> stream. (I'm ignoring for the moment that gRPC is heavily coupled to
>> HTTP/2. Since things like gRPC-web exist perhaps there is a way of
>> avoiding that coupling.)
>>
>> Has anyone else encountered these use cases, and if so, what do you
>> think about the various proxying options available?
>>
>> Best,
>> Chris
>>
>> [1] https://datatracker.ietf.org/doc/draft-ohai-chunked-ohttp/
>> --
>> Ohai mailing list
>> Ohai@ietf.org
>> https://www.ietf.org/mailman/listinfo/ohai
>>
>>
>> --
>> Ohai mailing list
>> Ohai@ietf.org
>> https://www.ietf.org/mailman/listinfo/ohai
>>
>>
>> --
>> Ohai mailing list
>> Ohai@ietf.org
>> https://www.ietf.org/mailman/listinfo/ohai
>>
>>
>> --
>> Ohai mailing list
>> Ohai@ietf.org
>> https://www.ietf.org/mailman/listinfo/ohai
>>
>
>