Re: Identify streams unidirectional vs bidirectional based on stream ID

[Jana Iyengar <[e-mail]>]

I'd like to merge #872 <https://github.com/quicwg/base-drafts/pull/872>. If
I don't hear any serious issues I'll merge it tomorrow.

On Mon, Oct 16, 2017 at 11:56 PM, Jana Iyengar <[e-mail]> wrote:

> On Mon, Oct 16, 2017 at 6:41 PM, Praveen Balasubramanian <
> [e-mail]> wrote:
>
>> I agree that this is adding complexity with four special cases, but I
>> would actually argue the other way: apps that want uni streams should be
>> able to use the existing bidi stream mechanism. An API for uni stream can
>> achieve this by closing the stream in one direction immediately after
>> opening it so that the very first packet for that stream signals that one
>> direction is closed and the other end knows that the stream is uni. Bidi
>> streams are a more general case and if we are going to support them (which
>> I think we should) then do we need any dedicated transport mechanisms for
>> uni streams?
>>
>
> There's not a lot of dedicated mechanisms; indeed you can implement uni as
> a special case of bidi streams. The issue was, as Ian summarized nicely
> here, that implicitly open streams became ambiguous. For instance if stream
> 15 was opened after stream 11,  the spec requires stream 13 to be opened
> implicitly, but should it be opened as a uni stream or a bidi one?
> Separating namespaces for uni/bidi means that only streams in that
> particular namespace are opened implicitly.
>
>
>
>>
>>
>> Thanks
>>
>>
>>
>> *From:* QUIC [mailto:[e-mail]] *On Behalf Of *Mikkel Fahnøe
>> Jørgensen
>> *Sent:* Monday, October 16, 2017 6:20 PM
>> *To:* Martin Thomson <[e-mail]>; Jana Iyengar <
>> [e-mail]>
>> *Cc:* Mike Bishop <[e-mail]>; Ian Swett <
>> [e-mail]>; [e-mail]; Hugues Fafard <[e-mail]>;
>> Roberto Peon <[e-mail]>
>>
>> *Subject:* Re: Identify streams unidirectional vs bidirectional based on
>> stream ID
>>
>>
>>
>> I’m just wondering why this approach was preferred as opposed to building
>> bidi on top of uni. I suppose this must have been discussed at the interim
>> and in part based on the recent experimental implementation of uni streams.
>>
>>
>>
>> A straightforward implementation may not have much difficulty in handling
>> the implementation either way but when you attempt to achieve low latency
>> and low memory foot print with shortest possible state life times including
>> coordination with application level buffers, then bidi streams are not very
>> fun to work with at the transport level. As become more agreeable after it
>> was concluded that teardown did not have to wait for FIN ACK and related,
>> but it is still a lot of complexity with potential bugs.
>>
>>
>>
>> One aspect of the complexity is that there are now four special cases
>> instead of two in the stream identifiers which must be dealt with in
>> several frame types. Some of this involves flow control per stream type
>> which bidi on top of uni would not support at the same granularity, but is
>> this really needed?
>>
>>
>>
>> Adding uni streams in addition to bidi streams is still a lot better than
>> not having uni streams because they solve problems such as async messaging
>> at high volume and partial reliability via throw away streams. However,
>> they do nothing to simplify the transport with the current proposal.
>>
>>
>>
>> Kind Regards,
>>
>> Mikkel Fahnøe Jørgensen
>>
>>
>>
>> On 17 October 2017 at 02.49.38, Martin Thomson ([e-mail])
>> wrote:
>>
>> Those low order bits are important. No need to rush (it's a
>> relatively small changeset).
>>
>> On Tue, Oct 17, 2017 at 10:48 AM, Jana Iyengar <[e-mail]> wrote:
>> > Based on what I'm seeing on the PR and on this thread, I think there's
>> > general agreement on this strategy to go forward. I would like to merge
>> this
>> > PR soon, modulo a new revision to address some open comments. (There's
>> > currently a question about whether to use the high or the low-order
>> bits for
>> > this, which should also get resolved before merging.)
>> >
>> > I'm asking folks to please review and speak up if they have
>> disagreements
>> > with the direction or with any specifics.
>> >
>> > I'll be away from connectivity for a week, so unless there are
>> unresolved
>> > issues then, I'd like to merge this in a week when I return. Martin can
>> > merge this sooner if there's only agreement.
>> >
>> > On Mon, Oct 16, 2017 at 8:15 AM, Ian Swett <[e-mail]>
>> wrote:
>> >>
>> >> I believe the motivation for separate stream limits is to put a
>> reasonable
>> >> limit on the number of incoming streams you'll accept. Otherwise, an
>> >> application could use a very large number of one type(ie: uni) of
>> stream and
>> >> almost none of another(ie: bidi), which would allow them to suddenly
>> open a
>> >> huge number of the other type(ie: bidi) of stream if they wanted.
>> >>
>> >> If we were using H2 style max open streams we could do it either way,
>> but
>> >> I think keeping them separate is probably best anyway.
>> >>
>> >> On Mon, Oct 16, 2017 at 11:01 AM, Roberto Peon <[e-mail]> wrote:
>> >>>
>> >>> In H2, the even/odd thing was just an easy way to encode a unique ID
>> per
>> >>> stream while indicating direction.
>> >>> This proposal seems to do the same thing, and that makes sense to
>> >>> me—instead of even-odd, we have mod-4-offset.
>> >>>
>> >>> I’d ask why bother to have separate stream limits. You could still
>> just
>> >>> have the one limit for the overall space.
>> >>> With H2, that was part of the point of putting it there (though it
>> was
>> >>> implit)—it was all part of one space, and thus the space was singly
>> >>> countable!
>> >>>
>> >>>
>> >>>
>> >>> -=R
>> >>>
>> >>>
>> >>>
>> >>> From: QUIC <[e-mail]> on behalf of Ian Swett
>> >>> <[e-mail]>
>> >>> Date: Friday, October 13, 2017 at 2:05 PM
>> >>> To: Mike Bishop <[e-mail]>
>> >>> Cc: Mikkel Fahnøe Jørgensen <[e-mail]>, "[e-mail]"
>> >>> <[e-mail]>, Hugues Fafard <[e-mail]>
>> >>>
>> >>>
>> >>> Subject: Re: Identify streams unidirectional vs bidirectional based
>> on
>> >>> stream ID
>> >>>
>> >>>
>> >>>
>> >>> In many ways, they are 4 independent spaces, but Ryan pointed out to
>> me
>> >>> that both for the purposes of draft text and implementation it's very
>> useful
>> >>> to have a single unique identifier for a stream, since otherwise you
>> have to
>> >>> talk about/pass around (StreamID, Type) everywhere instead of just
>> StreamID.
>> >>>
>> >>>
>> >>>
>> >>> Keeping one stream ID avoids creating two of all the stream specific
>> >>> frames(STOP_SENDING, RST_STREAM, STREAM_FRAME, MAX_DATA_OFFSET, etc),
>> one
>> >>> for each stream space.
>> >>>
>> >>>
>> >>>
>> >>> Using the two most significant bits would eliminate the usefulness of
>> any
>> >>> variable length integer encoding(current or Martin's new proposal),
>> so using
>> >>> the lower bits makes sense.
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Oct 13, 2017 at 4:56 PM, Mike Bishop
>> >>> <[e-mail]> wrote:
>> >>>
>> >>> Indeed. I almost think it makes more sense to say that there are four
>> >>> independent stream spaces, each with their own numbers. That does
>> mean that
>> >>> each side will need to separately manage Max Stream ID for two
>> spaces, but
>> >>> that seems more reasonable than the alternative. In fact, this PR
>> already
>> >>> implies that the limits for unidirectional and bidirectional streams
>> are
>> >>> handled separately, which gets really weird when the IDs are
>> interleaved
>> >>> with each other.
>> >>>
>> >>>
>> >>>
>> >>> In fact, this plays well with the variable-length encoding idea,
>> since
>> >>> the maximum size of a Stream ID would be 62 bits. They could then be
>> stored
>> >>> locally as a 64-bit value with the “extra” bits indicating which
>> space they
>> >>> refer to.
>> >>>
>> >>>
>> >>>
>> >>> From: QUIC [mailto:[e-mail]] On Behalf Of Mikkel Fahnøe
>> >>> Jørgensen
>> >>> Sent: Friday, October 13, 2017 1:04 PM
>> >>> To: [e-mail]; Hugues Fafard <[e-mail]>
>> >>> Subject: Re: Identify streams unidirectional vs bidirectional based
>> on
>> >>> stream ID
>> >>>
>> >>>
>> >>>
>> >>> Especially with the new variable length encoding proposal, storing
>> the
>> >>> identifiers early makes sense, as you only have to inspect the first
>> byte.
>> >>> But it gets complicated, because you don’t want to use 8 bytes for
>> some
>> >>> stream types and you don’t want the bits at some random location in a
>> >>> decoded 8 byte representation either.
>> >>>
>> >>>
>> >>>
>> >>> Regardless of bit placement, encoding state using bits is not ideal
>> when
>> >>> considering variable length encoding because you now use two bits for
>> length
>> >>> and two bits for stream type, leaving only 16 different streams
>> before the
>> >>> stream identifier needs to use two bytes, and so forth.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> Kind Regards,
>> >>>
>> >>> Mikkel Fahnøe Jørgensen
>> >>>
>> >>>
>> >>>
>> >>> On 13 October 2017 at 21.50.06, Hugues Fafard ([e-mail])
>> wrote:
>> >>>
>> >>> I see, you used the 2 least significant bits to encode that
>> >>> information. When only using one bit to indicate the direction, it
>> >>> boiled down to odd/even which more or less made sense, but when using
>> >>> 2 bits that breaks apart.
>> >>>
>> >>> Why not use the most significant bits instead? That way, getting the
>> >>> 'clean' stream ID without the flags would be as simple doing a `&
>> >>> 0x3F...FF` instead of bit-shifting things around. Similarly, encoding
>> >>> the stream ID would just entail `| 0x?0...00` to set the appropriate
>> >>> bits and also avoid bit-shifts.
>> >>>
>> >>> Regards,
>> >>> Hugues
>> >>>
>> >>> On 2017-10-13 20:51, Ryan Hamilton wrote:
>> >>> > As I understand the discussions at the interim, we've decided to
>> >>> > allow streams to be unidirectional or bidirectional. I support this
>> >>> > idea and was motivated to write up an idea I've been thinking about
>> >>> > for a while. In the same way that we use a bit in the stream ID to
>> >>> > identify the client vs server nature of the stream, we can use a
>> >>> > bit to identify the directionality. This divides the stream ID
>> >>> > space neatly into 4 distinct spaces (much like the 2 two we have
>> >>> > today: client/server). In addition this means that a stream ID
>> >>> > always uniquely identifies a single stream, and frames which
>> >>> > mention stream ID (STREAM, MAX_STREAM_ID, MAX_STREAM_DATA,
>> >>> > RST_STREAM, etc) do not need to explicitly convey the
>> >>> > directionality as fields in those frames (or implicitly based on
>> >>> > the directionality of the frames themselves).
>> >>> >
>> >>> > https://github.com/quicwg/base-drafts/pull/872
>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fbase-drafts%2Fpull%2F872&data=02%7C01%7Cpravb%40microsoft.com%7C7fd2013c69494e8d202a08d514fd326e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636438000073300849&sdata=ou8Qaukn6GgIa0bsWPVCWrTLkutNaW85THUQfjLduys%3D&reserved=0>
>> >>> >
>> >>> > Cheers,
>> >>> >
>> >>> > Ryan
>> >>>
>> >>>
>> >>
>> >>
>> >
>>
>>
>