Re: [Taps] RFC 6458 etc. in draft-welzl-taps-transports

[Michael Welzl <[e-mail]>]

Hi,

Thanks Karen for your long email and helpful comments!

I re-order it a bit and cut some parts to get my answer more structured:


First, to get this straight in the discussion:


>>> A side note about TCP, because Karen made a comment about the TCP API
>> too:
>>> a similar logic applies here, irrespective of whether the API is old
>>> or
>>> not: I think we should cover whatever a system claiming to "implement
>>> the protocol in accordance with the spec" has to cover. Going down the
>>> path of looking at actual socket API implementations is dangerous in
>>> that we end up in "only implemented here, not there" - land. We want a
>>> minimal set of mechanisms that are (or at least really should be! for
>>> that, what else can we use as a basis than our own recommendations?!)
>> available everywhere.
> 
> [Karen Elisabeth Egede Nielsen] Yes I agree completely ! The difficult
> exercise is where from
> to gather this minimal set of mechanisms in a qualified manner.

So I think we really do need to rely on RFCs.


1)  TCP:
-------------

I think there is a misunderstanding about draft-welzl-taps-transports:


> On 04 Nov 2015, at 02:27, Karen Elisabeth Egede Nielsen <[e-mail]> wrote:
> 
> HI,
> 
> As a general comment then I believe that when describing what is supported
> by TCP/SCTP (or UDP) as standard then it does not suffice to look into
> IETF RFCs.
> One need at least to relate to the *basic functions* of the POSIX/Berkeley
> socket api standard.
> 
> My understanding of the TCP API, for example, is that while RFC793 did
> specify an abstract API for TCP, then
> the true defacto standard for the socket api is the Berkeley socket api
> from .?. around 1990.
> Not saying the different socket api standards don't differ and that there
> is *one* standard to look at.
> But making is be RFC793 rather then what emerged as defacto in the 1990's
> seems a "bit odd" to me.
> 
> Especially since we have RFCs RFC1122 dating back to 1989 already
> clarifying part of RFC793 (namely the PUSH bit)
> And one, much more recent, RFC6093, clarifying the urgent bit.


I only used RFC4960 for SCTP in this -00 version, but I did try to use all of the TCP spec as far as possible... never easy with all these RFCs - but at least I constantly checked what I'm doing against the Roadmap RFC 7414.
So I did include RFC 1122 and RFC 6093. Specifically, the text at the end of Pass 1, under "Excluded Services", says the following about URG:

***
   The 'send' and 'receive' commands include usage of an "URGENT"
   mechanism, which SHOULD NOT be implemented according to [RFC6093]
   and is therefore not described here.  This also concerns the notification
   "Urgent pointer advance" in the ERROR_REPORT described in 
   Section 4.2.4.1 of [RFC1122].
***

and about PUSH:

***
The 'receive'
   command can (under some conditions) yield the status of the PUSH flag
   according to [RFC0793], but this TCP functionality is made optional
   in [RFC1122] and hence not considered here.  Generally, section
   4.2.2.2 of [RFC1122] says that PUSH on send calls MAY be implemented,
   which could be a reason not to consider it here.  However, the text
   then explains that "an interactive application protocol must set the
   PUSH flag at least in the last SEND call in each command or response
   sequence", and most implementations provide some option to cause a
   behavior that is in some way similar to PUSH.  Therefore PUSH is
   described as a part of SEND here.
***

We can debate these decisions, but I did base them on RFC 1122 and RFC 6093.

More about PUSH:

> [Karen Elisabeth Egede Nielsen]
> 
> The PUSH bit as settable by application was recognized as being optional
> by RFC1122 (1989).
> 
> Enforcing the PUSH bit set via Nagle off, does not mean that one can
> control the PUSH function (one do get Nagle off at the same time).

Technically, I don't see a connection between Nagle and PUSH. It's kind of obvious though that if you want small messages to be delivered fast, you don't want Nagle and you do want PUSH, but still these are separate functions.
This sounds exactly like the type of confusion we may end up with if we follow actual API implementations and not what the spec says!


> Possible certain implementations have certain tweeks so that one may
> control the PUSH bit from ULP. If that is consider to be the
> Defacto standard, even if RFC1122 says that it is optional, then I do not
> disagree, but is it so ?

See my interpretation above: I think RFC 1122 is a bit ambiguous here, once saying it MAY be implemented, then saying "an interactive application protocol must set the PUSH flag ...."  ... though this isn't a capitalized MUST.


> The PUSH bit makes the data be available for read by the application (when
> in-line) but I am not sure this, as defacto, makes
> the data be pushed to the application in most common api's exposed to
> applications today. In some implementations, a socket becomes readable
> based on the amount of data that has arrived correlated with when the
> application has asked to be awaken (in number of bytes), but the PUSH flag
> makes no difference. Possibly that is a deficiency of such
> implementations. Not sure ?
> I know that in some implementations then a socket becomes readable only if
> the receive buffer is full or the PUSH flag has arrived.

Yes, it's certainly implementation-dependent. The idea of PUSH is that, when not using it, the receiving process doesn't need to be woken up every time a new handful of bytes arrive.


> Perhaps I should ask in tcpm, if TCP implementations generally pushes the
> data to the application (or notifies the application) when the PUSH bit is
> seen ? Or perhaps I will get an overwhelming - YES SURE !! - response this
> to email, and then at least  the receive part of the PUSH is then inline
> with RFC1122/RFC793.

It's a mechanism of TCP, it's described in the spec, and it comes with some SHOULD's and MAY's. I think one can create a very efficient TCP implementation that completely ignores PUSH, but, depending on your system, it can also be helpful. Based on my analysis above, I thought it makes sense to assume that at least the sender can set it  (I think it's not explicit in today's APIs but implicit... I did take a look (just out of curiosity, not breaking my own rules when writing the document!  ;-)  ) and I think I found it in the BSD API or in some older version thereof).

See, that's the point of trying to rely only on RFCs: we end up with a view of what really should be implemented by the APIs out there. If it isn't, well, that's bad, but then it's really a breach of the spec.



2)  SCTP:
--------------


>>> 2) Reason two, more serious: RFC 6458 is Informational, and my
>>> understanding was that this is just one description of one API
>>> implementation, not a recommendation or even prescription of what all
>>> SCTP implementations must provide. However, my decision for
>>> draft-welzl-taps-transports was to *exclude* things that are only
>>> optional to implement - I don't think we want to end up with a TAPS
>>> system that provides services that, alas, on Operating System XY are
>>> not available because here only a subset of the API is implemented.
>>> Therefore I went with a minimal set of functions that I thought we can
>>> assume are implemented everywhere (well, at least in every system that
>>> claims to "follow the spec").  Can we assume that every system that
>>> claims to implement SCTP in accordance with the spec fully implements
> RFC
>> 6458?
>>> 
>> GF: From a TSVWG Chair perspective, beware here...  *ALL* more recent
>> IETF SCTP API work from TSVWG is INFO.  Each SCTP RFC is expected to
> have
>> an informative section that describes the API together with the
> normative
>> protocol spec. That is not because there are expected to be alternatives
> to
>> choose from:  It's because, as I understand, the IETF is not the formal
>> standards body for specification of such normative APIs.
>>> 
> [Karen Elisabeth Egede Nielsen] Not only that. RFC6458 was around as a
> tsvwg  wg draft
> long before RFC4960 work was started. My understanding is that while going
> from RFC2960 to RFC4960 one did
> not bother "too much" with updating the API section of RFC2960 -->
> RFC4960, as the finer API details were being
> worked on in this wg draft.
> 
> However I was not personally involved in IETF SCTP back then, so I like to
> ask others to confirm this.
> 
> Something else:
> 
> The implementation that I have worked with was initiated in the early 2000
> and at that time we
> looked for RFC2960bis (RFC4960) for protocol specification and the socket
> api for api features.
> Yes there are particulars of the RFC6458 which represent implementations
> aspects. For example
> our API implementation implements certain calls in the format they were
> specified in ver 14/15 of the socket api from
> draft from around 2007  and not the exact form the calls have in RFC6458
> (version 30-some of the draft) from 2011,
> but the features available as controllable has not changed much during the
> years. And it is the features available,
> not the implementation aspects that we are focused on in Taps.

I agree, I don't care much about the call format


> NB:
> But I have to admit that in our SCTP API implementation we implements the
> basic POSIX api functions of TCP/IP for SCTP simply
> when they makes as much sense for SCTP as they do for TCP/IP. For example
> DSCP is settable via a socket option per association,
> even if I think this cannot be found in RFC6458 nor in RFC4960 (which is a
> bit of a surprise to me).


Hm, this is about the only piece from RFC 6458 that I did cite in draft-welzl-taps-transports, just as a comment, because I found it disturbingly incoherent that TCP provides this but it's missing in RFC 4960:

***
   o  Change DSCP
      Protocols: TCP
      Comments: This is described to be changeable for SCTP too in
      [RFC6458].
***

In RFC 6458, "uint8_t  spp_dscp;" is a part of the per-adress parameters, struct sctp_paddrparams.


> So my view
> on the basic API functions of SCTP (as for TCP) is influenced by this.
> Which I have to recognize.

Hm. This statement makes me feel a bit uneasy...


So, if we have consensus that RFC 6458 reflects the services (not necessarily the precise call format) that are supposed to be provided by any SCTP implementation, I'm happy to include it. Let's hear some more voices perhaps...


Cheers,
Michael