RE: Split error codes in two

["Lubashev, Igor" <[e-mail]>]

Thinking about this a bit more, it seems that the key difference between "tcp socket" and "two pipe pairs" models is independence of the send and receive directions.

In "two pipe pairs", send and receive directions are completely independent (they are just two unidirectional streams with some syntactic sugar that allows them to be opened and closed together, if needed).  To be faithful to this model, we'd need to support each direction getting closed independently by either endpoint.

For "tcp socket", the two directions are not independent, and streams in both directions are reset if some data sent in either direction cannot be delivered (including when the receiver decided it does not want that data).  To be faithful to this model, we'd need transport to send RST_STREAM in response to RST_STREAM automatically, not because it needs to reset the other direction (the other direction is already semantically reset) but just for flow control accounting.  And there is also no STOP_SENDING, as this is just asking for a complete reset.

I am not yet seeing what "middle ground" can be here, while maintaining a coherent model/philosophy behind the choices.

From: Lubashev, Igor [mailto:[e-mail]]
Sent: Wednesday, September 13, 2017 1:27 AM
To: [e-mail]; [e-mail]
Subject: RE: Split error codes in two

This can be accomplished in either "tcp socket" or "pipe pairs" model, given a sufficiently large buffer and an ability of the API to poll on "EOS received" before consuming all buffered bytes (not normally a part of traditional APIs).

-----Original Message-----
From: Christian Huitema [[e-mail]]
Received: Wednesday, 13 Sep 2017, 12:57AM
To: [e-mail]<mailto:[e-mail]> [[e-mail]]
Subject: Re: Split error codes in two

There is another abstraction -- streams as messages. Application pushes a message that is the entire content of the stream, and receives a message once the entire stream has been received. That's the model used by DNS over QUIC.

On 9/12/2017 9:23 PM, Lubashev, Igor wrote:
This is a very good idea to decide what we want to model streams on.  Starting with well-known, simple abstractions seems like the best path to adoption.

There are two such abstractions that come to mind:


  1.  "TCP Stream sockets".  These only support signaling "orderly" closing of the write direction.  Closing of the read direction can only be done via close(), but that may leave the write direction in an undefined state (writing will complete, unless there was unread data in the read buffer or any data is received after close - see RFC 1122 4.2.2.13 $3).

I think the current spec allows for QUIC API to emulate TCP stream socket abstraction.  Current spec  also adds signaling "abort" closing of the write direction (RST_STREAM does not require an immediate RST_STREAM in the opposite direction), although there was some talk of removing this feature.


  1.  "A POSIX pipe pair", as Mike is proposing (one pair in each direction).  Connected pipes are very clean ('symmetric') abstractions, and they communicate close events (you close the 'write' pipe, 'read' pipe get EOS; you close the 'read' pipe, 'write' pipe gets an exception).



With read-direction close signaling, however, it is important to not to cause often-unnecessary STOP_SENDING frames sent for every common-case stream.close(). I think that could be done via one of the following ways:
a.       Unidirectional streams. Send STOP_SENDING on close() by the receiver (if the steam is open).
b.       API convention: "only send STOP_SENDING on stream.inputStream().close()". On stream.close(), only send STOP_SENDING if there is unread data or new data comes in later.
c.       Actually signal that read direction is closed in stream.close() by defining STREAM-with-FIN frame as "close both write and read directions" (common case).  Have separate frame types for WRITE_CLOSE, WRITE_ABORT (aka RST_STREAM), and READ_ABORT (aka STOP_SENDING).


From: Mike Bishop [mailto:[e-mail]]
Sent: Tuesday, September 12, 2017 8:04 PM
To: Jana Iyengar <[e-mail]><mailto:[e-mail]>; Martin Thomson <[e-mail]><mailto:[e-mail]>
Cc: Philipp S. Tiesel <[e-mail]><mailto:[e-mail]>; Subodh Iyengar <[e-mail]><mailto:[e-mail]>; Nick Harper <[e-mail]><mailto:[e-mail]>; QUIC WG <[e-mail]><mailto:[e-mail]>; Victor Vasiliev <[e-mail]><mailto:[e-mail]>
Subject: RE: Split error codes in two

Jana and I were discussing this today, and part of the difference comes down to how you conceive of the API a QUIC implementation will expose.  My mental model has been that a stream/stream-pair consists of an InputStream and an OutputStream; you can read on one, you can write on the other.  For the write stream, you can write your data to completion and close the stream, or you can decide to abort writing.  The wire expressions of these are FIN and RST_STREAM, respectively.  For the read stream, you can read the incoming data to the end or you can abort reading and discard the remainder of the stream; the wire expression of this would be flow control updates and STOP_SENDING, respectively.  On the other side, receipt of a RST_STREAM surfaces as a read error and receipt of a STOP_SENDING surfaces as a write error.  (Jana commented that in this mental model it might be clearer to call the frames WRITE_ABORT and READ_ABORT.)

If we take Martin's proposition that streams only close by application action, there would never be a write error surfaced to the application unless the underlying transport went away entirely.  Instead, it's the responsibility of the application protocol to communicate (on another stream, in this case) that you should abort writing, even if you've already written to completion, and then to do so.  That implies that the application needs the ability to return to streams on which it's done writing, in case it's later asked to go back and reset them.

Much of our discussions around unidirectional streams, stream state transitions, etc. including the question of whether STOP_SENDING belongs at the transport layer, stem from the fact that we all have different implicit mental models of the functional surface QUIC exposes.  Perhaps we should take a step back and agree on that first, as I believe Christian has suggested in the past?

From: Jana Iyengar [mailto:[e-mail]]
Sent: Friday, September 8, 2017 5:34 PM
To: Martin Thomson <[e-mail]<mailto:[e-mail]>>
Cc: Subodh Iyengar <[e-mail]<mailto:[e-mail]>>; Mike Bishop <[e-mail]<mailto:[e-mail]>>; Philipp S. Tiesel <[e-mail]<mailto:[e-mail]>>; QUIC WG <[e-mail]<mailto:[e-mail]>>; Nick Harper <[e-mail]<mailto:[e-mail]>>; Victor Vasiliev <[e-mail]<mailto:[e-mail]>>
Subject: Re: Split error codes in two

On Thu, Sep 7, 2017 at 7:50 PM, Martin Thomson <[e-mail]<mailto:[e-mail]>> wrote:
On Fri, Sep 8, 2017 at 9:42 AM, Jana Iyengar <[e-mail]<mailto:[e-mail]>> wrote:
> I don't think it makes sense to design an app protocol that doesn't send a
> RST in response to a RST.

I've been told not to invoke this particular demon, but you just
invoked the Unidirectional streams problem.  I think that we get there
because of this meme that says that data in the one direction is
somehow necessarily bound to data in the other direction.  That's an
entirely constructed notion.  A useful construct at times, certainly,
but that's not the point here.

I think that's the most common use of a RST -- to close both directions.

So I disagree.  There are many protocols in which you send messages
(== streams) in one direction but not another.  One of the ways you
get into a unidirectional state in the current draft is to end one
side.  A FIN is only one way to do that, a unidirectional RST can be
faster and even superior in other ways.  The HTTP use case clearly
illustrates that.

FIN is the primary way in which half-close is achieved, which is what you're talking about. The HTTP use case is a complete one-off -- STOP_SENDING is an odd enough use case that I'm always left wondering who actually uses it.

Also, as Igor observes, we don't require a reciprocal RST_STREAM in
the current draft.

Yes, we don't. Recall that the RST_STREAM was, prior to the change that doesn't require reciprocity, a bidirectional signal, which was simplified to a unidirectional one, with the general expectation that apps would close the other side when they receive this signal from QUIC. It's surely going to be true of HTTP. It's possible for an app to receive a RST and then keep sending, but that is a use case I haven't seen in practice.




--

Christian Huitema