Re: Implicitly opened streams and exposing stream IDs

Marten Seemann <martenseemann@gmail.com> Tue, 03 April 2018 07:10 UTC

Return-Path: <martenseemann@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E2DB51201FA for <quic@ietfa.amsl.com>; Tue, 3 Apr 2018 00:10:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level:
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3eY9ALyKIa1P for <quic@ietfa.amsl.com>; Tue, 3 Apr 2018 00:10:45 -0700 (PDT)
Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 44B00120047 for <quic@ietf.org>; Tue, 3 Apr 2018 00:10:45 -0700 (PDT)
Received: by mail-io0-x235.google.com with SMTP id m83so20717687ioi.8 for <quic@ietf.org>; Tue, 03 Apr 2018 00:10:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=S8OyK9ujcEawfHSz24/647e1Y4C5T1mAM4ZFQ/s20Hs=; b=tHaRzO5XFwOqvPne+PpYtEtuOxj0UjeFH3wTjTFjsLUqXwD9Uw0Vfhl7CdFnOD76HA +/sN1Bw/Kf1peUQF8IyydHWlTt1S4LPXfpuDW0qPBkgrJFG6zO2nxgfJ6PiCX7uLVI9Y CYW6xhGlkJ0FWTQutxus+ayZZ4I//XeBCLvaTli0oYDM5uX+FAKR4Vko9r1XdZiehTvm CCZyoH9IYFned40ob8+IPxxVoFhS5qlDEjjKe/jIqeM/orEIrpFyx5PLPruQFUPnjIl5 CuMp5yvB60GCBhof9/GSqYE4F6RNLN1Db38tUTb84R7lI26dYUAQ9ZexYJDYtZ5naeCA oz/g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=S8OyK9ujcEawfHSz24/647e1Y4C5T1mAM4ZFQ/s20Hs=; b=WU1zJ+GOmrD8ps0oUO7gTAtO4IyePiZL/dCSr36lSaQ9mO4sdWEzDtfEraaUoaO5h6 TPF7X0ZjviE7VSI5FxHTstQhpE/mBeoTGLdiMlNEgWto3dWCtUrXkN+KqQf0AA8vOL6a AdKpScpoO+c+8TGW6zgfNZxhcjUj/cF5uU5ynGuXq3uGMBXPD5OIQgQl4feLNjvpUH3K ffDsObs40+qPjFJ/kX5zHKR4AihJuRZQIlQr67SkA5dn2gjfKKxs37KhRA4JPw7pjXJT QanbieypWf8Uuq8PFAYuhF6X+m8vYBpCCHvhJkChLBIrGN4fySDxswd1Y7mbLfhb+BH+ BoBw==
X-Gm-Message-State: ALQs6tDgOIqS4DeUsEARKUQIEoxhYsgdXtTuyj3WuTU5KixHD+3YId6O +OHHkWDFVkx5LvnxavrgM61E6l11gjMtp1yznK4=
X-Google-Smtp-Source: AIpwx4+px2TUK6n8d2BWOAzEKBR0oYXpJjDSwDwQa34H+Putvh4KBPA55GHcG1SkKnAVwnwse3OYO9aPefQ0c+k6adA=
X-Received: by 10.107.180.136 with SMTP id d130mr10927079iof.129.1522739444318; Tue, 03 Apr 2018 00:10:44 -0700 (PDT)
MIME-Version: 1.0
References: <CAOYVs2qp3L-dTdFfBNDQT0Q=nCu+6Ew3gmF=0GMS2vVw1JfWCg@mail.gmail.com> <7CF7F94CB496BF4FAB1676F375F9666A3BB0D858@bgb01xud1012>
In-Reply-To: <7CF7F94CB496BF4FAB1676F375F9666A3BB0D858@bgb01xud1012>
From: Marten Seemann <martenseemann@gmail.com>
Date: Tue, 03 Apr 2018 07:10:33 +0000
Message-ID: <CAOYVs2qb+FmrC1GssCNrWvce0d=c_o4kii361vahoraNEZO=Zg@mail.gmail.com>
Subject: Re: Implicitly opened streams and exposing stream IDs
To: Lucas Pardue <Lucas.Pardue@bbc.co.uk>
Cc: QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c06b27aaa07d30568ec6781"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/n-V7bZ54rococri9XRfJtDMNNps>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Apr 2018 07:10:48 -0000

Hello,

Sure, the application *could* deal with it by starting every stream with
some kind of header, to signal what kind of stream type this is. I'm not
sure if I like this solution, since it creates additional complexity, and
it is only necessary because we removed a useful feature from the spec.
What did we actually gain from removing implicit stream opening from the
protocol? As far as I can see, we were able to relax the requirement
"endpoints MUST open the send side of streams for each type in order" to
"endpoints SHOULD open the send side of streams for each type in order".
This might seem nice from a conceptual viewpoint, but honestly, I don't
really see why anyone would actually want to do this. I think the main
argument against the MUST was that it's not enforceable anyway, but this
applies to a lot of other MUSTs in the spec as well (e.g. that flow control
limits can't be decreased).

The fix for this is straightforward: We should REQUIRE a peer to open
streams in order. How the receiver handles out of order streams then is an
implementation and API decision. I see two ways that an implementation
could reasonably deal with this:

   1. Return streams in the order they are received, e.g. return N+4 before
   N, if packets are reordered.
   2. Return streams ordered by stream ID, i.e. return first N and then N+4
   if a frame for N+4 is received.

This way, we wouldn't need to reintroduce implicitly opened streams, but
leave this up to implementations. I should have named this thread
differently, if only I had realized this earlier ;)

Regards,
Marten

On Mon, Apr 2, 2018 at 7:05 PM Lucas Pardue <Lucas.Pardue@bbc.co.uk> wrote:

> Hi Marten,
>
> Would Stream headers fix this problem? I.e. anything that requires special
> behaviour other than "bulk data" has some bytes of of magic at the start of
> the stream.
>
> Regards
> Lucas
> ________________________________________
> From: QUIC [quic-bounces@ietf.org] on behalf of Marten Seemann [
> martenseemann@gmail.com]
> Sent: 02 April 2018 12:43
> To: QUIC WG
> Subject: Implicitly opened streams and exposing stream IDs
>
> Recently, the implicit opening of streams (i.e. that when a receiver
> receives a frame for stream N+4, it can assume that stream N was already
> opened, but the packet might have been reordered) was removed from the
> draft. I think this change has some consequences that we haven't discussed
> so far.
>
> For the purpose of writing some pseudocode, I'm assuming a QUIC API that
> provides an AcceptStream() method, but the conclusions will be the same for
> a callback-based API.
>
>   *   If QUIC has implicit stream opening, AcceptStream() would return the
> streams in order (and if a frame opening stream N+4 is received before
> stream n is opened, AcceptStream() will first return stream N and then
> stream N+4).
>   *   Without implicit stream opening, AcceptStream() just returns
> whatever stream is received first. Streams might be returned in arbitrary
> order, if the peer doesn't open streams consecutively or if packets are
> reordered.
>
> Now imagine an application protocol where the first unidirectional stream
> opened by the client is a control stream, and all higher unidirectional
> streams are data streams. The application on the server side needs to find
> out which stream is the control stream, because it needs to be handled
> separately.
>
> With implicit stream opening, the server code would be:
>
>     control_stream = AcceptStream() // is guaranteed to open the first
> stream
>     // handle the control stream
>      while true:
>          stream = AcceptStream()
>          // handle the data stream
>
> and without implicit stream opening:
>
>     while true:
>         stream = AcceptStream()
>         if stream.ID() == kControlStreamID:
>             // handle the control stream
>         else:
>             // handle the data stream
>
> In this case, after accepting a stream, we first have to check the stream
> ID, since there's no guarantee if the control stream will actually be
> received first.
>
> For this stream mapping, it seems like the removal of implicitly opened
> streams implies that QUIC has to expose stream IDs to the application
> layer. I'm not sure if this was intended when making the change, especially
> since we're considering to change HQ such that it doesn't rely on QUIC
> stream IDs any more.
> We only manage to avoid the problem described here in our HTTP mapping,
> because the HTTP control streams are unidirectional and the request streams
> are bidirectional, and can therefore be told apart by their directionality.
> However, as a general transport protocol, other applications built on top
> of QUIC will have to find some way to deal with it.
>
>
> -----------------------------
> http://www.bbc.co.uk
> This e-mail (and any attachments) is confidential and
> may contain personal views which are not the views of the BBC unless
> specifically stated.
> If you have received it in
> error, please delete it from your system.
> Do not use, copy or disclose the
> information in any way nor act in reliance on it and notify the sender
> immediately.
> Please note that the BBC monitors e-mails
> sent or received.
> Further communication will signify your consent to
> this.
> -----------------------------
>