Re: Identify streams unidirectional vs bidirectional based on stream ID

[Ian Swett <[e-mail]>]

On a high level, I 'think' the working group has agreed on three things:
 1) The transport needs to support bidirectional streams (Prague)
 2) The approach of adding an association stream ID in the first few bytes
of the stream adds more complexity than we'd like. (Ekr's prototype in
Seattle)
 3) There is a lot of interest in having unidirectional streams as a first
class transport feature. (I can't call this consensus, but there's
certainly lots of interest)

This heads us towards an approach where another bit somewhere in the stream
frame indicates whether it's unidirectional or bidirectional.  One option
was my original PR#656 <https://github.com/quicwg/base-drafts/pull/656>,
but that would require changes to remove implicitly opened streams(ie: I
receive stream ID 7, indicating 5 must have been opened) from the current
QUIC transport.  Ryan's PR creates 4 separate namespaces for different
types of streams instead of the current two namespaces.  There a few ways
to move bits around, but I think most end up functionally very similar
either to my PR or Ryan's.


On Mon, Oct 16, 2017 at 9:41 PM, Praveen Balasubramanian <
[e-mail]> wrote:

> I agree that this is adding complexity with four special cases, but I
> would actually argue the other way: apps that want uni streams should be
> able to use the existing bidi stream mechanism. An API for uni stream can
> achieve this by closing the stream in one direction immediately after
> opening it so that the very first packet for that stream signals that one
> direction is closed and the other end knows that the stream is uni. Bidi
> streams are a more general case and if we are going to support them (which
> I think we should) then do we need any dedicated transport mechanisms for
> uni streams?
>
>
>
> Thanks
>
>
>
> *From:* QUIC [mailto:[e-mail]] *On Behalf Of *Mikkel Fahnøe
> Jørgensen
> *Sent:* Monday, October 16, 2017 6:20 PM
> *To:* Martin Thomson <[e-mail]>; Jana Iyengar <
> [e-mail]>
> *Cc:* Mike Bishop <[e-mail]>; Ian Swett <
> [e-mail]>; [e-mail]; Hugues Fafard <[e-mail]>;
> Roberto Peon <[e-mail]>
>
> *Subject:* Re: Identify streams unidirectional vs bidirectional based on
> stream ID
>
>
>
> I’m just wondering why this approach was preferred as opposed to building
> bidi on top of uni. I suppose this must have been discussed at the interim
> and in part based on the recent experimental implementation of uni streams.
>
>
>
> A straightforward implementation may not have much difficulty in handling
> the implementation either way but when you attempt to achieve low latency
> and low memory foot print with shortest possible state life times including
> coordination with application level buffers, then bidi streams are not very
> fun to work with at the transport level. As become more agreeable after it
> was concluded that teardown did not have to wait for FIN ACK and related,
> but it is still a lot of complexity with potential bugs.
>
>
>
> One aspect of the complexity is that there are now four special cases
> instead of two in the stream identifiers which must be dealt with in
> several frame types. Some of this involves flow control per stream type
> which bidi on top of uni would not support at the same granularity, but is
> this really needed?
>
>
>
> Adding uni streams in addition to bidi streams is still a lot better than
> not having uni streams because they solve problems such as async messaging
> at high volume and partial reliability via throw away streams. However,
> they do nothing to simplify the transport with the current proposal.
>
>
>
> Kind Regards,
>
> Mikkel Fahnøe Jørgensen
>
>
>
> On 17 October 2017 at 02.49.38, Martin Thomson ([e-mail])
> wrote:
>
> Those low order bits are important. No need to rush (it's a
> relatively small changeset).
>
> On Tue, Oct 17, 2017 at 10:48 AM, Jana Iyengar <[e-mail]> wrote:
> > Based on what I'm seeing on the PR and on this thread, I think there's
> > general agreement on this strategy to go forward. I would like to merge
> this
> > PR soon, modulo a new revision to address some open comments. (There's
> > currently a question about whether to use the high or the low-order bits
> for
> > this, which should also get resolved before merging.)
> >
> > I'm asking folks to please review and speak up if they have
> disagreements
> > with the direction or with any specifics.
> >
> > I'll be away from connectivity for a week, so unless there are
> unresolved
> > issues then, I'd like to merge this in a week when I return. Martin can
> > merge this sooner if there's only agreement.
> >
> > On Mon, Oct 16, 2017 at 8:15 AM, Ian Swett <[e-mail]> wrote:
> >>
> >> I believe the motivation for separate stream limits is to put a
> reasonable
> >> limit on the number of incoming streams you'll accept. Otherwise, an
> >> application could use a very large number of one type(ie: uni) of
> stream and
> >> almost none of another(ie: bidi), which would allow them to suddenly
> open a
> >> huge number of the other type(ie: bidi) of stream if they wanted.
> >>
> >> If we were using H2 style max open streams we could do it either way,
> but
> >> I think keeping them separate is probably best anyway.
> >>
> >> On Mon, Oct 16, 2017 at 11:01 AM, Roberto Peon <[e-mail]> wrote:
> >>>
> >>> In H2, the even/odd thing was just an easy way to encode a unique ID
> per
> >>> stream while indicating direction.
> >>> This proposal seems to do the same thing, and that makes sense to
> >>> me—instead of even-odd, we have mod-4-offset.
> >>>
> >>> I’d ask why bother to have separate stream limits. You could still
> just
> >>> have the one limit for the overall space.
> >>> With H2, that was part of the point of putting it there (though it was
> >>> implit)—it was all part of one space, and thus the space was singly
> >>> countable!
> >>>
> >>>
> >>>
> >>> -=R
> >>>
> >>>
> >>>
> >>> From: QUIC <[e-mail]> on behalf of Ian Swett
> >>> <[e-mail]>
> >>> Date: Friday, October 13, 2017 at 2:05 PM
> >>> To: Mike Bishop <[e-mail]>
> >>> Cc: Mikkel Fahnøe Jørgensen <[e-mail]>, "[e-mail]"
> >>> <[e-mail]>, Hugues Fafard <[e-mail]>
> >>>
> >>>
> >>> Subject: Re: Identify streams unidirectional vs bidirectional based on
> >>> stream ID
> >>>
> >>>
> >>>
> >>> In many ways, they are 4 independent spaces, but Ryan pointed out to
> me
> >>> that both for the purposes of draft text and implementation it's very
> useful
> >>> to have a single unique identifier for a stream, since otherwise you
> have to
> >>> talk about/pass around (StreamID, Type) everywhere instead of just
> StreamID.
> >>>
> >>>
> >>>
> >>> Keeping one stream ID avoids creating two of all the stream specific
> >>> frames(STOP_SENDING, RST_STREAM, STREAM_FRAME, MAX_DATA_OFFSET, etc),
> one
> >>> for each stream space.
> >>>
> >>>
> >>>
> >>> Using the two most significant bits would eliminate the usefulness of
> any
> >>> variable length integer encoding(current or Martin's new proposal), so
> using
> >>> the lower bits makes sense.
> >>>
> >>>
> >>>
> >>> On Fri, Oct 13, 2017 at 4:56 PM, Mike Bishop
> >>> <[e-mail]> wrote:
> >>>
> >>> Indeed. I almost think it makes more sense to say that there are four
> >>> independent stream spaces, each with their own numbers. That does mean
> that
> >>> each side will need to separately manage Max Stream ID for two spaces,
> but
> >>> that seems more reasonable than the alternative. In fact, this PR
> already
> >>> implies that the limits for unidirectional and bidirectional streams
> are
> >>> handled separately, which gets really weird when the IDs are
> interleaved
> >>> with each other.
> >>>
> >>>
> >>>
> >>> In fact, this plays well with the variable-length encoding idea, since
> >>> the maximum size of a Stream ID would be 62 bits. They could then be
> stored
> >>> locally as a 64-bit value with the “extra” bits indicating which space
> they
> >>> refer to.
> >>>
> >>>
> >>>
> >>> From: QUIC [mailto:[e-mail]] On Behalf Of Mikkel Fahnøe
> >>> Jørgensen
> >>> Sent: Friday, October 13, 2017 1:04 PM
> >>> To: [e-mail]; Hugues Fafard <[e-mail]>
> >>> Subject: Re: Identify streams unidirectional vs bidirectional based on
> >>> stream ID
> >>>
> >>>
> >>>
> >>> Especially with the new variable length encoding proposal, storing the
> >>> identifiers early makes sense, as you only have to inspect the first
> byte.
> >>> But it gets complicated, because you don’t want to use 8 bytes for
> some
> >>> stream types and you don’t want the bits at some random location in a
> >>> decoded 8 byte representation either.
> >>>
> >>>
> >>>
> >>> Regardless of bit placement, encoding state using bits is not ideal
> when
> >>> considering variable length encoding because you now use two bits for
> length
> >>> and two bits for stream type, leaving only 16 different streams before
> the
> >>> stream identifier needs to use two bytes, and so forth.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> Kind Regards,
> >>>
> >>> Mikkel Fahnøe Jørgensen
> >>>
> >>>
> >>>
> >>> On 13 October 2017 at 21.50.06, Hugues Fafard ([e-mail])
> wrote:
> >>>
> >>> I see, you used the 2 least significant bits to encode that
> >>> information. When only using one bit to indicate the direction, it
> >>> boiled down to odd/even which more or less made sense, but when using
> >>> 2 bits that breaks apart.
> >>>
> >>> Why not use the most significant bits instead? That way, getting the
> >>> 'clean' stream ID without the flags would be as simple doing a `&
> >>> 0x3F...FF` instead of bit-shifting things around. Similarly, encoding
> >>> the stream ID would just entail `| 0x?0...00` to set the appropriate
> >>> bits and also avoid bit-shifts.
> >>>
> >>> Regards,
> >>> Hugues
> >>>
> >>> On 2017-10-13 20:51, Ryan Hamilton wrote:
> >>> > As I understand the discussions at the interim, we've decided to
> >>> > allow streams to be unidirectional or bidirectional. I support this
> >>> > idea and was motivated to write up an idea I've been thinking about
> >>> > for a while. In the same way that we use a bit in the stream ID to
> >>> > identify the client vs server nature of the stream, we can use a
> >>> > bit to identify the directionality. This divides the stream ID
> >>> > space neatly into 4 distinct spaces (much like the 2 two we have
> >>> > today: client/server). In addition this means that a stream ID
> >>> > always uniquely identifies a single stream, and frames which
> >>> > mention stream ID (STREAM, MAX_STREAM_ID, MAX_STREAM_DATA,
> >>> > RST_STREAM, etc) do not need to explicitly convey the
> >>> > directionality as fields in those frames (or implicitly based on
> >>> > the directionality of the frames themselves).
> >>> >
> >>> > https://github.com/quicwg/base-drafts/pull/872
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F872&data=02%7C01%7Cpravb%40microsoft.com%7C7fd2013c69494e8d202a08d514fd326e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636438000073300849&sdata=ou8Qaukn6GgIa0bsWPVCWrTLkutNaW85THUQfjLduys%3D&reserved=0>
> >>> >
> >>> > Cheers,
> >>> >
> >>> > Ryan
> >>>
> >>>
> >>
> >>
> >
>
>