Re: [Taps] Review of draft-trammell-taps-post-sockets-03

[Tommy Pauly <[e-mail]>]

Hi Michael,

Just a couple initial notes that may help:

- The version diff you should look at is between -01 and -03. -02 is the same as -03, but had a typo.

- You've mentioned previously that you thought that Post requires both peers to use that API. That is absolutely not the case in any way. Having implemented Post myself, I am only communicating to "legacy" servers that know nothing about Post. I think this fundamental understanding needs to be cleared up before coming to any conclusions. We can discuss during the WG.

- The point of the API is to give the shape of the application interaction, which is why you find it general. I believe that many of the protocol-specific options like you mention (disabling Nagle, retransmissions, etc) don't belong to this main abstract API draft, but to a different document that goes into how to configure options that can be used for setups like TCP/UDP. Again, in my implementation of Post, all of these options exist as part of the Configuration object mentioned in the draft. However, as I'll discuss in the WG, I believe that the general shape of the API needs to be defined to be more-forward looking than just what we've specified with the minset for TCP/UDP/SCTP. Essentially, these become a set of Configuration options that can be used, but as transport protocols continue to evolve (and when we need more protocol-specific options), we need to be able to expand. Certain aspects of the minset, like the connection state management, are of course general and common enough to be part of the highest level of API description.

Thanks,
Tommy

> On Nov 11, 2017, at 9:43 AM, Michael Welzl <[e-mail]> wrote:
> 
> Dear all,
> 
> Because of the planned request for draft-trammell-taps-post-sockets-03 to become a WG item, I gave the draft another close read and decided to write a review based on this. I hope it's useful.
> 
> To me, post-sockets aims in the right direction - I like a number of things  about it: the introduction of higher-layer concepts such as the carrier and association; I suspect that the long-term association management could also be implemented one-sided, which is nice. I like the idea of "forking" a carrier, which would clearly translate into doing multi-streaming "under the hood" whenever that's possible (side note: from the list on pages 7/8, it seems that *only* "Listener" can be forked. Why not a Source? This seems odd to me). Finally, obviously I also agree that being message-oriented is the right way to go.
> 
> 
> On the negative side, I still see several problems here (actually almost nothing has changed from --02 to -03), listed below:
> 
> The biggest problem I see is that this has very limited usage scenarios, as it requires a post-socket system on both sides (there's quite a bit of functionality in there that won't work otherwise - to give an illustrative example: "When a Carrier is forked, its corresponding Carrier at the remote endpoint receives a fork request, .."). I find it particularly disappointing because, in my opinion, it wouldn't take much to make this able to fall-back to TCP such that e.g. a post-sockets client could talk to an existing TCP server. Anyway, what IS this falling back to? If you add functionality both on the sender and receiver side, you require a post-sockets system on both sides - and then you could just as well consistently run over UDP or TCP or HTTP/2 or whatever else you want, without any fall-backs (and as this develops new transport functionality, we're then really moving away from the TAPS charter). I think we must better understand what types of fall-backs are envisioned here.
> 
> The document talks about "asynchronous reception", as if this was a feature that would only be attainable with a novel approach, and/or a message-based transport. I see having a callback-based API and a socket-style API as largely orthogonal from the protocol below: one can of course also write a callback-based API over TCP... the real problem is to know after how much received data the receiving app should be notified, by whatever means - that's just a general dilemma, and not a new problem, as the PUSH bit in RFC 793 shows. Messages may seem to make this easier, but then you have to start wondering how large can messages be, how we should handle partial messages... e.g., you may need a signal from the receiver-side TAPS system to the app to say "you have stored half a message, but please throw this data block away now, you'll never get the rest". You have some discussion of partial messages in this draft, and we've been around that block with SCTP. IMO, the right approach is to lea
> ve all of that out: define a short max limit, and let an application take care of splitting into smaller messages itself (since you have message dependencies, why do you need to have large messages too, and not just smaller sub-messages that all sequentially depend on another?). This also makes prioritization more meaningful.
> 
> Generally, the draft just leaves many questions unanswered at this stage, as it stays at a rather high level and lists most services just as examples, saying that more could be possible. If it's that vague, how should anybody implement such a system? Here's a list of transport features (from minset) that we will lose if we don't offer them to an application, and that I didn't see mentioned in post-sockets (sorry if I missed some!). Some of them may seem a bit odd and unimportant, others less so (this is from appendix A2, where "T" and "U" mean "can be supported with a fall-back to TCP and UDP, respectively):
> 
>  o  T,U: Specify number of attempts and/or timeout for the first
>     establishment message
>  o  T: Change timeout for aborting connection (using retransmit limit
>     or time value)
>  o  T: Suggest timeout to the peer
>  o  T,U: Disable Nagle algorithm
>  o  T,U: Notification of Excessive Retransmissions (early warning
>     below abortion threshold)
>  o  T,U: Specify DSCP field
>  o  T,U: Notification of ICMP error message arrival
>  o  T,U: Set Cookie life value
>  o  T,U: Disable checksum when sending
>  o  T,U: Disable checksum requirement when receiving
>  o  T,U: Specify checksum coverage used by the sender
>  o  T,U: Specify minimum checksum coverage required by receiver
>  o  T,U: Specify DF field
>  o  T,U: Get max. transport-message size that may be sent using a non-
>     fragmented IP packet from the configured interface
>  o  T,U: Get max. transport-message size that may be received from the
>     configured interface
>  o  T,U: Obtain ECN field
>  o  T,U: Enable and configure a "Low Extra Delay Background Transfer"
>  o  T,U: Request not to delay the acknowledgement (SACK) of a message
>  o  T,U: Notification that the stack has no more user data to send
> 
> 
> From this list, a few functions that I would consider important are:
> 
> - DSCP, and enabling and configuring LEDBAT-like behavior: these two could be lumped together in a higher-level request, but they're not just a priority. Sure, your message "niceness" could be translated into a DSCP choice, but I mean that an app may want to be able to control more (e.g. do a general per-carrier configuration for low latency or LBE, for example). Speaking of LEDBAT, post-sockets also doesn't say anything about whether the transport is congestion controlled or not (which translates into: does the app have to worry about it?). Another congestion control thing: access to the ECN flag in case of UDP should also be covered in some form.
> 
> - Notification that the stack has no more user data to send: this is the TCP_NOWAIT_LOWAT function. Hmmm... wait!  At first I thought the binding to an .OnAcked() event handler can't be done with TCP, but if the post-sockets layer would intelligently use TCP_NOWAIT_LOWAT, then you could pull this off.
> 
> - Get max. transport-message size that may be sent using a non-fragmented IP packet from the configured interface: this is useful to avoid fragmentation overhead, and
> 
> - Specify DF field => if we fall back to UDP, we need this in order to do PMTUD.
> 
> 
> ... and then there's also the need for early configuration of some things to guide protocol choice, which I wrote as a decision tree in draft-ietf-taps-minset-00. While I agree with your idea of specifying a limited set of protocols to use - we do the same with policy in NEAT - that can't be all: a TAPS system should probably prevent, e.g., first picking UDP and then getting a request for reliability. That brings us back to the question about what you're planning to fall back to, of course.
> 
> 
> I hope this was helpful,
> 
> cheers,
> Michael
> 
> _______________________________________________
> Taps mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/taps