Re: [Taps] RFC 6458 etc. in draft-welzl-taps-transports

[Karen Elisabeth Egede Nielsen <[e-mail]>]

HI Michael,

Thanks a lot.
Please see inline below.

BR, Karen

> -----Original Message-----
> From: Michael Welzl [mailto:[e-mail]]
> Sent: 4. november 2015 20:06
> To: Karen Elisabeth Egede Nielsen <[e-mail]>
> Cc: <[e-mail]> Fairhurst <[e-mail]>;
> [e-mail]
> Subject: Re: [Taps] RFC 6458 etc. in draft-welzl-taps-transports
>
> Hi,
>
> Thanks Karen for your long email and helpful comments!
>
> I re-order it a bit and cut some parts to get my answer more structured:
>
>
> First, to get this straight in the discussion:
>
>
> >>> A side note about TCP, because Karen made a comment about the TCP
> >>> API
> >> too:
> >>> a similar logic applies here, irrespective of whether the API is old
> >>> or
> >>> not: I think we should cover whatever a system claiming to
> >>> "implement the protocol in accordance with the spec" has to cover.
> >>> Going down the path of looking at actual socket API implementations
> >>> is dangerous in that we end up in "only implemented here, not there"
> >>> - land. We want a minimal set of mechanisms that are (or at least
> >>> really should be! for that, what else can we use as a basis than our
> >>> own recommendations?!)
> >> available everywhere.
> >
> > [Karen Elisabeth Egede Nielsen] Yes I agree completely ! The difficult
> > exercise is where from to gather this minimal set of mechanisms in a
> > qualified manner.
>
> So I think we really do need to rely on RFCs.
>
>
> 1)  TCP:
> -------------
>
> I think there is a misunderstanding about draft-welzl-taps-transports:
>
>
> > On 04 Nov 2015, at 02:27, Karen Elisabeth Egede Nielsen
> <[e-mail]> wrote:
> >
> > HI,
> >
> > As a general comment then I believe that when describing what is
> > supported by TCP/SCTP (or UDP) as standard then it does not suffice to
> > look into IETF RFCs.
> > One need at least to relate to the *basic functions* of the
> > POSIX/Berkeley socket api standard.
> >
> > My understanding of the TCP API, for example, is that while RFC793 did
> > specify an abstract API for TCP, then the true defacto standard for
> > the socket api is the Berkeley socket api from .?. around 1990.
> > Not saying the different socket api standards don't differ and that
> > there is *one* standard to look at.
> > But making is be RFC793 rather then what emerged as defacto in the
> > 1990's seems a "bit odd" to me.
> >
> > Especially since we have RFCs RFC1122 dating back to 1989 already
> > clarifying part of RFC793 (namely the PUSH bit) And one, much more
> > recent, RFC6093, clarifying the urgent bit.
>
>
> I only used RFC4960 for SCTP in this -00 version, but I did try to use
all of the
> TCP spec as far as possible... never easy with all these RFCs - but at
least I
> constantly checked what I'm doing against the Roadmap RFC 7414.
> So I did include RFC 1122 and RFC 6093. Specifically, the text at the
end of
> Pass 1, under "Excluded Services", says the following about URG:
>
> ***
>    The 'send' and 'receive' commands include usage of an "URGENT"
>    mechanism, which SHOULD NOT be implemented according to [RFC6093]
>    and is therefore not described here.  This also concerns the
notification
>    "Urgent pointer advance" in the ERROR_REPORT described in
>    Section 4.2.4.1 of [RFC1122].
> ***
>
[Karen Elisabeth Egede Nielsen] Yes I am happy with this.
I may have more problems with the transport 07 draft in this context.
This comment I have given there.

Sorry for referring to it here in the context of your draft. It just
belonged
in the overall discussion.

> and about PUSH:
>
> ***
> The 'receive'
>    command can (under some conditions) yield the status of the PUSH flag
>    according to [RFC0793], but this TCP functionality is made optional
>    in [RFC1122] and hence not considered here.  Generally, section
>    4.2.2.2 of [RFC1122] says that PUSH on send calls MAY be implemented,
>    which could be a reason not to consider it here.  However, the text
>    then explains that "an interactive application protocol must set the
>    PUSH flag at least in the last SEND call in each command or response
>    sequence", and most implementations provide some option to cause a
>    behavior that is in some way similar to PUSH.  Therefore PUSH is
>    described as a part of SEND here.
> ***
>
[Karen Elisabeth Egede Nielsen] There is a very clear distinction in
between
capitalized Keywords and non-capitalized must's and should's in section
4.2.2.2 of rfc1122.

 I have no problem with assuming that TCP implementations follow
           MUST set the PSH bit in
            the last buffered segment (i.e., when there is no more
            queued data to be sent).

I have no problem with assuming that applications, if the MAY option is
supported at the api, s
Is assumed to set the PUSH flag at the occasions described in RFC1122.

But I do have a problem  with the following text - repeated from your
paragraph above;:

>  and most implementations provide some option to cause a
>    behavior that is in some way similar to PUSH.  Therefore PUSH is
>    described as a part of SEND here.

You are referring to "most implementations provide some option"

What do you mean by "most implementations" ?

What do you mean by "option" ?

My problem particularly with "the option" part, is that is to me in the
context of
an API description sounds as if this "option" is a function in the API
(read an "API option")
 that the ULP can invoke.


> We can debate these decisions, but I did base them on RFC 1122 and RFC
> 6093.
>
> More about PUSH:
>
> > [Karen Elisabeth Egede Nielsen]
> >
> > The PUSH bit as settable by application was recognized as being
> > optional by RFC1122 (1989).
> >
> > Enforcing the PUSH bit set via Nagle off, does not mean that one can
> > control the PUSH function (one do get Nagle off at the same time).
>
> Technically, I don't see a connection between Nagle and PUSH. It's kind
of
> obvious though that if you want small messages to be delivered fast, you
> don't want Nagle and you do want PUSH, but still these are separate
> functions.

[Karen Elisabeth Egede Nielsen] Perfect. I cound't agree more !!
I was trying to understand which "API option" that you might be referring
to,
and as I have had many (in my opinion confused) discussions with
certain TCP users which looking for the "API option" for setting of the
PUSH,
thought they had to use Nagle off for this. While in fact they didn't have
to do
anything because the stack will set the PUSH bit following the RFC1122
MUST.

> This sounds exactly like the type of confusion we may end up with if we
> follow actual API implementations and not what the spec says!
>
[Karen Elisabeth Egede Nielsen] GOOD. I am happy that we avoid this.
So could we loose the text about that the TCP API provides an option to
control the PUSH flag - :-) :-) !!
>
> > Possible certain implementations have certain tweeks so that one may
> > control the PUSH bit from ULP. If that is consider to be the Defacto
> > standard, even if RFC1122 says that it is optional, then I do not
> > disagree, but is it so ?
>
> See my interpretation above: I think RFC 1122 is a bit ambiguous here,
once
> saying it MAY be implemented, then saying "an interactive application
> protocol must set the PUSH flag ...."  ... though this isn't a
capitalized MUST.
>
[Karen Elisabeth Egede Nielsen] For what it is worth (if anything)  this
is not how I interpret the text. I understand
the text as saying that if the MAY option to set PUSH in the apu
is implemented then applications must do what the text says. But as
RFC1122 is not really specifying what applications should or must do.,
these musts or should are not  capitalized.

But I should assume that we could get the people working with RFC793bis in
tcpm
provide the "right" interpretation ?!


>
> > The PUSH bit makes the data be available for read by the application
> > (when
> > in-line) but I am not sure this, as defacto, makes the data be pushed
> > to the application in most common api's exposed to applications today.
> > In some implementations, a socket becomes readable based on the
> amount
> > of data that has arrived correlated with when the application has
> > asked to be awaken (in number of bytes), but the PUSH flag makes no
> > difference. Possibly that is a deficiency of such implementations. Not
> > sure ?
> > I know that in some implementations then a socket becomes readable
> > only if the receive buffer is full or the PUSH flag has arrived.
>
> Yes, it's certainly implementation-dependent. The idea of PUSH is that,
when
> not using it, the receiving process doesn't need to be woken up every
time a
> new handful of bytes arrive.
>
[Karen Elisabeth Egede Nielsen] Yes and certainly, as many TCP
implementations simply
always sets the PUSH flag following the MUST of rfc1122, section 4.2.2.2,

then the PUSH flag ends up being set on a lot of segments just because the
application is writing in
small chunks....

Or even worse - the PUSH flag  it is lost when the SEND buffer gets
congested,
which is terribly when the receiving side read operation has implemented
trust on the PUSH
for being waken up. This actually is something that we have seen resulting
in
outages (signaling network collapse)  even recently.


>
> > Perhaps I should ask in tcpm, if TCP implementations generally pushes
> > the data to the application (or notifies the application) when the
> > PUSH bit is seen ? Or perhaps I will get an overwhelming - YES SURE !!
> > - response this to email, and then at least  the receive part of the
> > PUSH is then inline with RFC1122/RFC793.
>
> It's a mechanism of TCP, it's described in the spec, and it comes with
some
> SHOULD's and MAY's. I think one can create a very efficient TCP
> implementation that completely ignores PUSH,
but, depending on your
> system, it can also be helpful.
[Karen Elisabeth Egede Nielsen] Perhaps - but it can also be very harmful
to put trust in the PUSH
flag, if the two ends to not agree on what controls it - the ULP or the
TCP SND buffer congestion level...

Based on my analysis above, I thought it makes
> sense to assume that at least the sender can set it  (I think it's not
explicit in
> today's APIs but implicit... I did take a look (just out of curiosity,
not breaking
> my own rules when writing the document!  ;-)  ) and I think I found it
in the
> BSD API or in some older version thereof).
>
[Karen Elisabeth Egede Nielsen] But the implicit is what gets us into the
confusion !
How is it implicitly implemented in API  implementations ?

(Despite that rfc1122 say that it need not be).

> See, that's the point of trying to rely only on RFCs: we end up with a
view of
> what really should be implemented by the APIs out there. If it isn't,
well,
> that's bad, but then it's really a breach of the spec.
>
[Karen Elisabeth Egede Nielsen] In this case then I am afraid that it is
the spec that allowed the
implementations to distroy the functions as it (rfc793 to rfc1122) has
given some freedom in implementations.
The uncertainty of which has made the function unreliable/not trustworthy.
(Or perhaps the function was never a good idea to begin with..)

>
>
> 2)  SCTP:
> --------------
>
>
> >>> 2) Reason two, more serious: RFC 6458 is Informational, and my
> >>> understanding was that this is just one description of one API
> >>> implementation, not a recommendation or even prescription of what
> >>> all SCTP implementations must provide. However, my decision for
> >>> draft-welzl-taps-transports was to *exclude* things that are only
> >>> optional to implement - I don't think we want to end up with a TAPS
> >>> system that provides services that, alas, on Operating System XY are
> >>> not available because here only a subset of the API is implemented.
> >>> Therefore I went with a minimal set of functions that I thought we
> >>> can assume are implemented everywhere (well, at least in every
> >>> system that claims to "follow the spec").  Can we assume that every
> >>> system that claims to implement SCTP in accordance with the spec
> >>> fully implements
> > RFC
> >> 6458?
> >>>
> >> GF: From a TSVWG Chair perspective, beware here...  *ALL* more recent
> >> IETF SCTP API work from TSVWG is INFO.  Each SCTP RFC is expected to
> > have
> >> an informative section that describes the API together with the
> > normative
> >> protocol spec. That is not because there are expected to be
> >> alternatives
> > to
> >> choose from:  It's because, as I understand, the IETF is not the
> >> formal standards body for specification of such normative APIs.
> >>>
> > [Karen Elisabeth Egede Nielsen] Not only that. RFC6458 was around as a
> > tsvwg  wg draft long before RFC4960 work was started. My understanding
> > is that while going from RFC2960 to RFC4960 one did not bother "too
> > much" with updating the API section of RFC2960 --> RFC4960, as the
> > finer API details were being worked on in this wg draft.
> >
> > However I was not personally involved in IETF SCTP back then, so I
> > like to ask others to confirm this.
> >
> > Something else:
> >
> > The implementation that I have worked with was initiated in the early
> > 2000 and at that time we looked for RFC2960bis (RFC4960) for protocol
> > specification and the socket api for api features.
> > Yes there are particulars of the RFC6458 which represent
> > implementations aspects. For example our API implementation
> implements
> > certain calls in the format they were specified in ver 14/15 of the
> > socket api from draft from around 2007  and not the exact form the
> > calls have in RFC6458 (version 30-some of the draft) from 2011, but
> > the features available as controllable has not changed much during the
> > years. And it is the features available, not the implementation
> > aspects that we are focused on in Taps.
>
> I agree, I don't care much about the call format
>
>
> > NB:
> > But I have to admit that in our SCTP API implementation we implements
> > the basic POSIX api functions of TCP/IP for SCTP simply when they
> > makes as much sense for SCTP as they do for TCP/IP. For example DSCP
> > is settable via a socket option per association, even if I think this
> > cannot be found in RFC6458 nor in RFC4960 (which is a bit of a
> > surprise to me).
>
>
> Hm, this is about the only piece from RFC 6458 that I did cite in
draft-welzl-
> taps-transports, just as a comment, because I found it disturbingly
> incoherent that TCP provides this but it's missing in RFC 4960:
>
> ***
>    o  Change DSCP
>       Protocols: TCP
>       Comments: This is described to be changeable for SCTP too in
>       [RFC6458].
> ***
>
> In RFC 6458, "uint8_t  spp_dscp;" is a part of the per-adress
parameters,
> struct sctp_paddrparams.
>
>
[Karen Elisabeth Egede Nielsen] Great. I missed it yesterday. Couldn't
understand it either.
Perhaps I can blame the jetlag...

> > So my view
> > on the basic API functions of SCTP (as for TCP) is influenced by this.
> > Which I have to recognize.
>
> Hm. This statement makes me feel a bit uneasy...
>

[Karen Elisabeth Egede Nielsen] Yes. :-(.

>
> So, if we have consensus that RFC 6458 reflects the services (not
necessarily
> the precise call format) that are supposed to be provided by any SCTP
> implementation, I'm happy to include it. Let's hear some more voices
> perhaps...
>
[Karen Elisabeth Egede Nielsen] Yes definitively.



>
> Cheers,
> Michael