Re: [Taps] API design: dependencies or buffer control?

[Tommy Pauly <[e-mail]>]

The code I work with does TCP_NOTSENT_LOWAT by default, so we have a fair amount of experience with it. If you're using sockets with TCP_NOTSENT_LOWAT, and you're doing asynchronous non-blocking socket operations, then you don't have the annoyance of getting these "empty" callbacks you're referring to—it just changes how aggressively the writable event fires, making it back off a bit.

With a Post-like API, having something directly like TCP_NOTSENT_LOWAT doesn't make much sense. Instead, the implementation may internally use that, but the equivalent application feedback is the completion that is returned based on the write messages. The timing of when the stack indicates when something has been written allows the application to understand the back-pressure properties of the transport, and if it able to generate or fetch data more slowly to match the back-pressure, it can. Otherwise, it can simply keep writing and the data will get enqueued within the library.

Dependencies between the messages that are being written, then, doesn't actually come into play much here. Dependencies are hints to the implementation and protocol stack of how to order work when scheduling to send (PRE-WRITE). TCP_NOTSENT_LOWAT or the completions to indicate that writes have completed are all about signaling back to the application to provide back-pressure (POST-WRITE).

Does that help clarify things?

Thanks,
Tommy

> On Nov 13, 2017, at 12:30 AM, Michael Welzl <[e-mail]> wrote:
> 
> Hi,
> 
> Another thought related to the post-sockets draft:
> 
> Post-sockets lets an application programmer define dependencies. That’s good, because dependencies exist, but it comes at the cost of complexity.
> My gut feeling tells me: if you have dependencies to take care of, it’s best to leave the data with the application and not hand it over to the transport at all until the last minute.
> 
> Controlling the buffer with something like TCP_NOTSENT_LOWAT would be a way for an application to only give the most recent, relevant data to the transport, and decide to throw away data on its own in case a preceding message was thrown away (which it might learn from transport).
> 
> I haven’t written code that uses TCP_NOTSENT_LOWAT myself, but I would guess that the trade-off in using this is: if you set this value low, you have more control over the buffer and can be more efficient in your decisions, but the disadvantage is that the transport permanently comes back to you and says “empty, empty, empty, ….” - which may be inconvenient / inefficient for the programmer.
> 
> If I’m right with all my musings here, then for an application that has block dependencies, the design should depend on what is easier to handle: defining priorities towards the transport layer on one side, or dealing with the transport calling us quickly all the time because its buffer is about to run empty, plus handling dependencies in the application on the other.
> 
> I don’t know - which one IS easier?
> 
> I’d love to see this discussed a little, and I’d also love to know if my interpretation of the trade-off in this email is even correct.
> 
> Cheers,
> Michael
> 
> _______________________________________________
> Taps mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/taps