[Taps] TAPS and higher-level APIs

[Michael Welzl <[e-mail]>]

Dear all,

Looking at this:

> On Nov 13, 2023, at 10:24 PM, Michael Welzl <[e-mail]> wrote:
> 
> +1 … thanks a whole lot to everyone, it’s been a fun ride!
> 
> For some nostalgia, more detailed history is here:  https://sites.google.com/site/transportprotocolservices/home <https://sites.google.com/site/transportprotocolservices/home>
… reminded me of something, and I think that now is perhaps a good moment to share this thought.  This should be more useful than nostalgia - it’s a note for the researchers out there: I think that in a post-TAPS world, we still have a big hole that needs filling (and it’s probably not in scope of the IETF). In my opinion, this hole is a low-hanging-fruit for researchers who are interested in this general space.

Here it is:

One of the major arguments against TAPS, back when we started this effort, was: “you want to change the socket API, but nobody writes applications against the socket API anymore”.
Well, one has to start somewhere, and sockets sit below the middleware or library that offers these APIs. I think we had enough reasons to start the effort - but the point is still valid: many applications *are* written over REST, or pub-sub, or what have you, and today won’t care much about TAPS services.

When thinking about how to combine such middleware layers with TAPS, there are the following possibilities:
1) the application is unchanged, the middleware or library below it uses TAPS, and somehow, some benefits arise (e.g., from dynamically selecting QUIC or TCP). Browsers do that today; other middleware layers could do it too. This would yield some benefits, but not necessarily everything.
2) the higher-layer API is changed too: many of the services that TAPS applications choose are truly related to the *application*, and hard or impossible for a middleware layer to “guess” - e.g., we have priorities between defined groups of Connections, a need for reliability with time limits, various services that translate into a DSCP choice, telling the API about bounds on the send and receive rate, control over checksums, ….

What would be needed, to make use of TAPS in such a scenario, is to take a decision, per middleware layer:
a) which of the TAPS services are best implemented transparently to the application, i.e. in the middleware, without changing the API that is being offered,
b) which services should be offered in the higher-level API,
c) which services should be ignored.

Let me get back to why the URL above was a reminder: at the BoF at IETF-89 in London, Martin Sustrik, who created the well-known ZeroMQ pub-sub middleware, gave a presentation on “transport agnostic middleware”. Here are the slides:
https://www.ietf.org/proceedings/89/slides/slides-89-taps-0.pdf <https://www.ietf.org/proceedings/89/slides/slides-89-taps-0.pdf>
We also have the conversation after this presentation in the minutes of the BoF, IMO this is worth reading:  https://www.ietf.org/proceedings/89/minutes/minutes-89-taps <https://www.ietf.org/proceedings/89/minutes/minutes-89-taps>
A part of his point was that pub-sub doesn’t need (or even want) 100% reliability - he had this example of distributing data where one subscriber is a small trading shop somewhere in Malaysia, and the distributor doesn’t want to wait for that small shop to pull the data. I believe that pub-sub middlewares today probably implement that sort of unreliability (or, rather: timed reliability) on top of the socket API… which is probably just less efficient than telling the socket API about it.

I got in touch with Martin back then because his interest in middleware + transport was obvious from another library he was working on at the time: nanomsg ( https://nanomsg.org/ <https://nanomsg.org/> ), meant to run over SCTP.

Anyway, the pub-sub case he talked about is just one example out of many that probably exist within pub-sub, and in other communication patterns.  My hope is that someone picks this up and thinks ahead, on: what can we do now, to let applications using higher layer APIs benefit from TAPS?

Cheers,
Michael