Re: [Taps] Review of draft-trammell-taps-post-sockets-03

[Michael Welzl <[e-mail]>]

> On Nov 13, 2017, at 8:07 AM, Tommy Pauly <[e-mail]> wrote:
> 
> 
> 
>> On Nov 11, 2017, at 2:09 PM, Michael Welzl <[e-mail] <mailto:[e-mail]>> wrote:
>> 
>> Hi,
>> 
>> In line:
>> 
>>> On Nov 11, 2017, at 11:58 AM, Tommy Pauly <[e-mail] <mailto:[e-mail]>> wrote:
>>> 
>>> 
>>> 
>>>> On Nov 11, 2017, at 10:36 AM, Michael Welzl <[e-mail] <mailto:[e-mail]>> wrote:
>>>> 
>>>> 
>>>>> On Nov 11, 2017, at 10:06 AM, Tommy Pauly <[e-mail] <mailto:[e-mail]>> wrote:
>>>>> 
>>>>> Hi Michael,
>>>>> 
>>>>> Just a couple initial notes that may help:
>>>>> 
>>>>> - The version diff you should look at is between -01 and -03. -02 is the same as -03, but had a typo.
>>>>> 
>>>>> - You've mentioned previously that you thought that Post requires both peers to use that API. That is absolutely not the case in any way. Having implemented Post myself, I am only communicating to "legacy" servers that know nothing about Post. I think this fundamental understanding needs to be cleared up before coming to any conclusions. We can discuss during the WG.
>>>> 
>>>> Oh?!  That would be fantastic!!  But I stumbled over a number of things that require a system on the other side, or they just couldn’t happen (like the “carrier forking” notification I quoted below - but there were more). If you say you can talk to a “legacy” server that knows nothing about Post, WHAT does that server speak? Legacy which protocol?  Maybe it’s just a matter of me thinking TCP, and you thinking TLS… dunno! Tell me   :)
>>> 
>>> So, since Post (and TAPS in general) isn't defining any new protocol features, we need to of course be compatible with all existing protocols. Certainly, some functionality is a bit degenerate in some cases.
>>> 
>>> Whatever the server speaks, the client needs to conform to. So, we use our Post implementation for raw TCP, raw UDP, TLS over TCP, HTTP/2 Streams over TLS over TCP, QUIC streams over UDP, etc. These are all standard server configurations, but Post offers a client application a single set of APIs to talk over any of these protocols.
>> 
>> Oh, great!  But then you can’t guarantee that a message arrives as a message in case of raw TCP, or you assume application-level framing. This is what I’ve been proposing, but the post-sockets draft reads entirely different from that.
> 
> Messages do indeed "always work", but they may be degenerate. There's no new protocols here, just ways of looking at them:
> 
> - UDP: One message per datagram
> - TCP: Entire stream is one big message that only ever ends when that half of the stream closes
> - TLS: Generally viewed like TLS, but can view one record as one message 
> - HTTP/QUIC: One message per request/response (so, framing)
> - Length-value framing on streams, as used by some protocols (like IKEv2 over TCP): one message per L-V frame
> Etc, etc.
> 
> That is what the draft does say in Section 2.2. Note how a TCP stream is the message in some cases.
> 
> A Message is the unit of communication between applications.
>    Messages can represent relatively small structures, such as requests
>    in a request/response protocol such as HTTP; relatively large
>    structures, such as files of arbitrary size in a filesystem; and
>    structures of indeterminate length, such as a stream of bytes in a
>    protocol like TCP.

I read this; how is this efficient?  Are you suggesting to open and close connections to signal that a message is over?
Otherwise, how can you know it is? The length of a message is determined by the application...


>>> In the case of "Carrier Forking", that's essentially what you say to the protocol stack when you want to open a new stream to the same place. So, when I call "fork Carrier", it turns into:
>>> - For raw UDP/TCP or TLS (or anything that is mono-streamed), open a new five-tuple to the same remote endpoint
>>> - For HTTP/2 and QUIC, open a new stream on the same connection
>> 
>> Sure, that’s what I thought it would. Now, with SCTP, you wouldn’t get a “fork request” equivalent on the other side: the client would just start using a new stream, and that’s it.  (“request” also indicates the possibility to say no, which also doesn’t exist in SCTP AFAIK, at least not in the API - this would be a “stream reset”).  Now, in QUIC, I don’t know exactly how that is, but I would have thought it’s the same - you just go ahead and use a new stream. No “fork request” on the other side - just a new stream (in post, carrier) being used.
> 
> The term "fork-request" is misleading, yes. It really should just say: "forking a carrier creates a new flow to the same Remote, which may be a new stream on a multiplexing protocol. The protocol will handle any required signaling to the remote to open a new stream, based on the protocol”.

Ok; the remote must not have a chance to refuse it, then, because the protocol may not really signal anything all before you get your first byte of data.


>>>>> - The point of the API is to give the shape of the application interaction, which is why you find it general. I believe that many of the protocol-specific options like you mention (disabling Nagle, retransmissions, etc) don't belong to this main abstract API draft, but to a different document that goes into how to configure options that can be used for setups like TCP/UDP. Again, in my implementation of Post, all of these options exist as part of the Configuration object mentioned in the draft. However, as I'll discuss in the WG, I believe that the general shape of the API needs to be defined to be more-forward looking than just what we've specified with the minset for TCP/UDP/SCTP. Essentially, these become a set of Configuration options that can be used, but as transport protocols continue to evolve (and when we need more protocol-specific options), we need to be able to expand. Certain aspects of the minset, like the connection state management, are of course general and common enough to be part of the highest level of API description.
>>>> 
>>>> It’s clear to me that we want a higher abstraction level than what the list from minset has - e.g., rather than a DSCP value, it would be better to specify general requirements (low latency or such) for a carrier. Rather than saying “disable Nagle”, we could also say “ low latency, even if it comes at the cost of some overhead” - we do such things in the NEAT API too. You’re focused on the interaction with the application (e.g. callback-based instead of traditional socket-style) - which is fine, but I think doesn’t have much to do with the actual protocol choice.
>>>> 
>>>> As for being more forward-looking, I wonder what the new transport features are that we’re missing out on (things that apps really see). I follow QUIC from a slightly too large distance (I simply run out of cycles there :(  ), but so far I’m not sure there’s anything we’d be missing  (but that’s maybe also because I’m not an HTTP/2 expert either). Except security of course, but there the argument is perhaps that it’s enough to consider falling back to TLS?
>>> 
>>> QUIC is using TLS 1.3, so it can give equivalent security properties to HTTP/2. Doing fallback between these is definitely an important feature. I think this flexibility is why the API needs to allow per-protocol configs.
>> 
>> Sure, but I think (and that was my point) this doesn’t change anything about the stuff above (security is in your separate document) - so what exactly are the “more forward looking” requirements except for these security needs?
> 
> I'm trying to distinguish certain aspects of the minset that are:
> a) fundamental to all transports, that we have reason to believe will be the case for all future transports in some fashion
> 	CONNECT, SEND, RECEIVE, CLOSE, etc
> b) derived from the protocols we happened to survey or based on current semantics (sockets), based on what exists
> 	Things I see in the NEAT draft like SET_MIN_CHECKSUM_COVERAGE, SET_TTL, SET_LOW_WATERMARK. Important to have, but often the values the application needs to set relate to which protocol might be used, and may have different requirements for future protocols.
> 
> So, there's a set of things in category (a) that should be stable as the main API go forward for quite a while, while things in category (b) may belong as configuration properties, in a list that may shrink or expand over time as new developments are made.

Hmm. Not sure I buy that e.g. set_min_checksum_coverage has any dependency on the protocol.  As for set_low_watermark, see my other email about that matter: the point is, the application should have some means to tune the size of the buffer below it, I believe.

Cheers,
Michael