Re: [alto] alto transport streams (Re: early reviews)

["Y. Richard Yang" <[e-mail]>]

Hi Spencer,

Thank you so much for the reviews from both you and Mark. We are working on
how to address the additional comments from you including the configuration
settings issues. We are discussing them and will respond soon.

Thanks again!
Richard

On Fri, Jul 22, 2022 at 12:27 PM Spencer Dawkins at IETF <
[e-mail]> wrote:

> I'd just like to note that I kept asking myself what Mark Nottingham would
> say, while thinking about my own review comments - I'm really glad to see
> that you're also talking to Mark!
>
> Best,
>
> Spencer
>
> On Thu, Jul 21, 2022, 23:00 Mark Nottingham <[e-mail]> wrote:
>
>> Hi Richard,
>>
>> I'm just going to give some impressions while reading this, FWIW.
>>
>> > On 22 Jul 2022, at 6:20 am, Y. Richard Yang <[e-mail]> wrote:
>> >
>> > Hi Mark, Martin, Spencer,
>> >
>> > First, thank you so much for the reviews. Sorry for the delay in
>> responding asap, due to summer and travel.
>> >
>> > One common comment we see from all three of you, in different texts, is
>> essentially on the service model of the current document (
>> https://datatracker.ietf.org/doc/draft-ietf-alto-new-transport/): how
>> much control on the ordering of the messages (Mark/Martin), in-order or not
>> (Spencer). Hence, after the meeting early morning today, we feel that we
>> should start a single thread to discuss this issue. We will send a summary
>> of our responses to each of your individual reviews later.
>> >
>> > To help make clear the problem and the current design, let's start by
>> summarizing the transport model of ALTO/SSE (RFC8895;
>> https://datatracker.ietf.org/doc/html/rfc8895). We will state the
>> problem as abstract as possible to see the essence of the problem and the
>> potential design choices:
>> > - P1: A server offers multiple resources R = {R[1], R[2], R[3], R[4],
>> ...}, where each R[i]i is represnted by a URI;
>>
>> Good so far; that's pretty much a description of every use of HTTP.
>>
>> > - P2: A client can request from the server the monitoring of a subset M
>> of the resources: M \subset R, e.g., M = {R[1], R[2], R[3]}; for example,
>> R[1] is a network map, R[2] is a cost map using the network map R[1], and
>> R[3] is an endpoint property map; for simplicity, assume the requested
>> resources numbered 1 to |M|
>>
>> So, in HTTP terms, that would be creating a new resource whose semantic
>> is "I am a monitor for  _this_ set of resources". The tricky party here is
>> how to realise that in terms of HTTP -- i.e., what does a GET response look
>> like?
>>
>> One answer would be a feed format like RSS or Atom. Do you support GET in
>> this fashion?
>>
>> > - P3: The server can push a sequence of incremental updates to the
>> client for each resource in M. Denote {U[i,1], U[i,2], ...} as the update
>> sequence from the server to the client for R[i] in M, where U[i,1] is the
>> first response for the requested resource R[i], U[i,2] is an incremental
>> update (patch) on top of U[i,1], U[i,3] is the incremental update on top of
>> U[i,2], ...
>>
>> Here's where I get more concerned. Server Push is unproven, and indeed
>> has been shown to be an anti-pattern for its originally intended use case.
>> Folks are still interested in it for API use cases (and I'd see this as one
>> of those), but it's still very wild west, with no real widespread
>> deployment experience that I'm aware of. See especially <
>> https://httpwg.org/specs/rfc9205.html#server-push>.
>>
>> The first question I'd ask here is whether polling a resource (like a
>> feed document) is sufficient. That pattern works very well with HTTP, and
>> is well-understood -- _much_ more so than Server Push.
>>
>> The downside, of course, is latency, but it's not unknown to deploy
>> applications with very high polling frequencies (e.g., 1/s). Have you
>> considered this approach? Would modifying it to long-polling (where the
>> client always keeps a request 'hanging' until the server has an update)
>> help? Keep in mind that with HTTP/2 and above, long-polling has very few
>> downsides.
>>
>> Another option would be to invert the relationship and have what's
>> currently the server open connections to the current client and PUT / PATCH
>> updates to them. Much more natural HTTP, but of course you need an identity
>> for the client and a clear path to it. This approach also has considerable
>> deployment experience (commonly known as 'webhooks').
>>
>> > - P3.1: For flexibility, each U[i,j] can choose its own encoding; for
>> example, U[1,1] is application/alto-networkmap+json, U[1,2] is
>> application/merge-patch+json; concrete examples please see the current
>> draft (search "Promised Stream 4" and "Promised Stream 6")
>>
>> Sure.
>>
>> > - P4: Consider the dependency among the information resources in P3:
>> {U[1,1], U[1,2], ...}, {U[2,1], U[2,2], ...}, and {U[3,1], ....}. There are
>> two types:
>> > - P4.1 We have that U[i,j+1] depends on U[i,j] due to incremental
>> updates---in the general case, the client needs to have received and
>> processed U[i,j] to apply U[i,j+1], unless U[i,j+1] is a snapshot (not an
>> incremental update).
>>
>> Right. The closest things that we have for managing this sort of thing in
>> HTTP today are conditional requests; e.g., If-Match.
>>
>> > - P4.2 It is possible that U[i', j'] depends on U[i, j], where i' != i:
>> for example, a cost map depends on the correct version of the network map.
>>
>> That's a purely application-level semantic, correct? I.e., resource A
>> isn't useful without resource B, and furthermore representation 1 of
>> resource A requires representation 4 of resource B to be interpreted
>> correctly. This is similar to issues that people face in caching CSS,
>> JavaScript and the like today -- generally it's solved by giving
>> representations with breaking changes different URLs, and referencing them
>> directly from their dependants. We've talked about other ways to solve
>> this, but that's current practice.
>>
>> > If one goes academic, there is a dependency graph that can describe the
>> dependencies.
>> >
>> > In RFC8895, since it is a single HTTP connection, all of the {U[i,j]}
>> are *linearized" by the server into a single sequence, and Section 6.7 of
>> RFC8895 specifies the linearization (serialization) requirements (P4.1 and
>> P4.2). As we know from concurrency control, linearization is strong and
>> leaves performance on the table. One of the motivations for the new
>> document is to take advantage of the more relaxed concurrency model allowed
>> by HTTP/2 and HTTP/3.
>> >
>> > So what are the design points that the design team has considered:
>> >
>> > - D1 (linearization): the server pushes all {U[i,j]} in a single
>> stream. Due to P3.1, there must be an internal structure to separate the
>> U[i,j] boundaries and different U[i,j] can have different media types,
>> leading back to RFC8895.
>>
>> I don't understand why it's necessary to do this. AFAICT there are no
>> ordering constraints created by state-changing operations; rather, you just
>> have dependencies that can be resolved once all of the updates are
>> available.
>>
>> For example, if you *are* going to stay with Server Push, I immediately
>> wonder what the value of having a monitor resource is, rather than just
>> having the appropriate resources push updates directly using their own
>> identity. Of course you still need some sort of subscription mechanism, but
>> that doesn't mean that the actual updates need to be coalesced into one
>> HTTP response.
>>
>> >
>> > - D2 (max concurrency): Each U[i,j] is sent in an independent
>> (concurrent) push stream (and hence can use its own media type as it is)
>> and let the application-layer handles dependency: ALTO has a build-in
>> dependency check for cross-resource (P4.2) and the sequence numbers of
>> incremental updates allow application (ALTO client) to figure out the
>> same-resource dependency (P4.1). The application (ALTO client) buffers all
>> streams to realize the ordering.
>>
>> Right, this seems more reasonable, although again I wonder about the use
>> of Push.
>>
>> > - D3 (max concurrency with server barrier): It is D2 but requires that
>> the server does not push U[i',j'] if there is a U[i,j], s.t., (1) U[i,j] is
>> still being sent by the server to the client and (2) U[i',j'] depends on
>> U[i,j].
>> >
>> > The intention of the current document is to reflect D3. Note that for
>> D3, the ALTO client still needs to handle the dependency correctly, as the
>> streams may be only buffered at the kernel, and not processed. But the
>> benefit of D3 over D2 is that it implements essentially some flow control,
>> to avoid a faster server overwhelming a slower client.
>>
>> The streams might be buffered anywhere in the handling chain. Given that
>> H2/H3 already have flow control, is this really necessary? Or are you
>> trying to manage buffer sizes 'above' the HTTP layer?
>>
>> > One design point that one may consider is
>> > - D4 (linearization of same resource and concurrency for different
>> resources): the U[i,j] of the same R[i] is sent in the same stream (and
>> hence linearized); this can be a reasonable design, but due to P3.1, it is
>> still not clean; see D1 discussion.
>>
>> Indeed.
>>
>> > I hope that the preceding has clarified the stream control issue that
>> the design faced. One can see that the issue can be considered as a generic
>> issue--what to specify when embedding one concurrency model (P4.1/P4.2)
>> into a given concurrency structure (HTTP/2 or HTTP/3). We took a quick look
>> at DoH. It looks that the dependency model of DoH might be simpler: there
>> is no same-resource (DNS resource type) due to no incremental updates (DNS
>> update model appears to be the idempotent overwrite model) or
>> cross-resource dependency. Please correct us if we have missed it.
>> >
>> > The current document tried to separate the issue into Sec. 8 (ALTO/H2
>> Stream Management). One way forward we can see is to (1) only specify that
>> each U[i,] is mapped to its own stream (to handle P3.1), and (2) not
>> specify the dependency among U[i,j] (i.e., specify as close as possible to
>> specify nothing) and let the client figure out the dependency at the
>> message; we add the requirement language such as "The client MUST check the
>> dependency ...;"
>>
>> Overall, this seems like a very awkward layering of an application onto
>> HTTP. I don't know enough about the underlying requirements to tell whether
>> a more natural approach is possible today, but I suspect it might be.
>>
>> Being able to get updates to the state of a set of resources is a pretty
>> common requirement, so I'd very much like to see this addressed in a
>> generic way that's both reusable for other applications and that works well
>> with the other parts of HTTP. I wrote a little more about this here: <
>> https://www.mnot.net/blog/2022/02/20/websockets>.
>>
>> Another thing that comes to mind is that what you're trying to do seems
>> to have _some_ overlap with what the BRAID folks are trying to do: <
>> https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-braid-http>.
>> I'd be very interested to hear if you thought it'd be helpful to have that
>> document move forward.
>>
>> Hope this helps,
>>
>>
>> --
>> Mark Nottingham   https://www.mnot.net/
>>
>