Re: Implicitly opened streams and exposing stream IDs
Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com> Tue, 03 April 2018 07:50 UTC
Return-Path: <mikkelfj@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DBF1412E053 for <quic@ietfa.amsl.com>; Tue, 3 Apr 2018 00:50:21 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.697
X-Spam-Level:
X-Spam-Status: No, score=-1.697 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sjXhosTyY74b for <quic@ietfa.amsl.com>; Tue, 3 Apr 2018 00:50:18 -0700 (PDT)
Received: from mail-io0-x232.google.com (mail-io0-x232.google.com [IPv6:2607:f8b0:4001:c06::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A6A91124205 for <quic@ietf.org>; Tue, 3 Apr 2018 00:50:18 -0700 (PDT)
Received: by mail-io0-x232.google.com with SMTP id l3so20873465iog.0 for <quic@ietf.org>; Tue, 03 Apr 2018 00:50:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=Ujp8zeDrv6zB1rLdiua90JrJPudt1NmIhL/rALLb5fw=; b=I27qjFTW9mn5gaOVbcDfutRoVv07mCoaiQyuVyllot1nFDGx44cnl+Nkstf1HknMMi +aP6BA1/rc5U+HtiDypiOKPc3kifIpMxOVSIJP01FaYZnEMn8RD7DQiOAucuzx+S7au8 LFfGkL2X70OOxvUebrXCsnOojxI07vEbM4iNg+GLmj+iEYFrtAR6IHPSjNcTGiCqBE8p MovxTeDG983wBl3d4GX0cobrMsg9gqaEh5OCzFqi+UQhSzp/hNbx0L7tCM92RVTl5P5N BRu8gYAoPiwJstgTlnxw3BYri7meVbE5S8dwBq5J5IllZHrMLeARaSmbikP4hk2j1URQ DP2Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=Ujp8zeDrv6zB1rLdiua90JrJPudt1NmIhL/rALLb5fw=; b=ggP/gqsSp9F7yA+umqyvxIWm6JwrbGyDVjuDKxUD3nkdpaaMPZVv9VSTTbfHxNAFA0 1KF5gDqFkGRTZ8DsXrZ7XmB3RvehdCagrUdC2hhcvFw3sed6VN/utAsbigfFUDS5E3hw 90iZhz7/MKMO20VB9GD8fktQZdVVo7+jyqILs2TDQH09ncy9Xz5iViCqVd0y+vzyLaqr VbI2Vkw6JazwqjyqtchoNA+EQPOWlK5wv2Ezc2giti+EPtJglVJBRv1Kb04vPqVPITk3 oWjF/HG2Q3le6i8Ng396YP0uj5wnsCXGLzP+SSLYTBggta7j3SSerRgwsBfnNjkfGgpM aETA==
X-Gm-Message-State: ALQs6tCDCXWmO26ReZzsHd5wV+Cd+2zy/oUl71k17q1ZQg85EbKtX7VB LDzGNm4J1ovX75gAsfnGg4gIMdTScSbLvXn3WevdZQ==
X-Google-Smtp-Source: AIpwx4/qKT4gzEVeg9AHDHqJQ1ACO6t2b9cfdf+LFpPXfsM0YThi/LPLO4NkDWxJUDTRxElqIh8D8uhHh2TZClkWZJo=
X-Received: by 10.107.6.163 with SMTP id f35mr12050953ioi.165.1522741817849; Tue, 03 Apr 2018 00:50:17 -0700 (PDT)
Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Tue, 3 Apr 2018 00:50:17 -0700
From: Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>
In-Reply-To: <CABkgnnWBZ0nRxoJB9XdqQ8JF6etAnCEpjT6c=2ZD76XcghismQ@mail.gmail.com>
References: <CAOYVs2qp3L-dTdFfBNDQT0Q=nCu+6Ew3gmF=0GMS2vVw1JfWCg@mail.gmail.com> <7CF7F94CB496BF4FAB1676F375F9666A3BB0D858@bgb01xud1012> <CAOYVs2qb+FmrC1GssCNrWvce0d=c_o4kii361vahoraNEZO=Zg@mail.gmail.com> <CABkgnnWBZ0nRxoJB9XdqQ8JF6etAnCEpjT6c=2ZD76XcghismQ@mail.gmail.com>
X-Mailer: Airmail (420)
MIME-Version: 1.0
Date: Tue, 03 Apr 2018 00:50:16 -0700
Message-ID: <CAN1APdfy=1Z494iK7bPzEKanbEC3fn=qWCuGKr_TKxvMxqQrKg@mail.gmail.com>
Subject: Re: Implicitly opened streams and exposing stream IDs
To: Martin Thomson <martin.thomson@gmail.com>, Marten Seemann <martenseemann@gmail.com>
Cc: Lucas Pardue <lucas.pardue@bbc.co.uk>, QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="001a113e06a2233d310568ecf595"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/73mn199mt3Lzg7mN3SixIqLU2eE>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Apr 2018 07:50:22 -0000
Say you deal with 10K outbound bidi streams. As Martin suggests, you can make this heavily concurrent and also reserve certain patterns for certain use. With implicit bi-directional streams you must be prepared to handle connection state for all 10K streams that is supposed to be started by you. You cannot create an error if you receive data, at least not at transport level, so you must open and track state. This in turn will limit you number of streams you are willing to handle at once, or force you to split streams into linked uni-streams. Further, there might be zero application logic that understands what data on such a stream means so when an unfortunate subsystem decides to open a stream, it has to ponder whatever the data already pending on that stream means. Opening strictly in order (whether required or not) reduces those problems, but it also severely constrains the application. For example you cannot send a stream id internally to a subsystem and ask it to produce data once ready to do so. You need to invent a complex mapping that is difficult to coordinate in a distributed system. It is much simpler to deal with the stream id’s directly. With multi-path you could even have multiple hosts managing a single connection, and if not, you could still do it with proper address mapping. So implicit streams and strict opening order might simplify very simple problems, but also make very hard problems even harder. Also, the entire idea of linearising QUIC seems to go against the spirit of independent non-blocking streams. Still, having a maximum stream ID is very useful in limiting the resources that must be managed. Mikkel On 3 April 2018 at 09.28.38, Martin Thomson (martin.thomson@gmail.com) wrote: Requiring in-order opening is harder than it sounds. Say you have a multithreaded application that is initiating requests. The usual process is something like: s = connection.OpenStream() s.Write(request.Headers.Encode()) That is, there is time between acquisition of the stream and its use. Do that with enough concurrency and you end up having to add late binding of stream identifiers to streams, which complicates the design of the stack considerably. It also complicates usage because now there is a period where s.Id() returns an undefined result. Alternatively, you could track the highest stream ID on which data has been sent and send an empty, offset 0 STREAM frame rather than create a gap. Or, you could decide that a MUST here is unenforceable (even more so than the flow control limits one) and just wantonly ignore that requirement if this race condition happens. While it is in theory possible to catch someone in a violation, it requires some interesting conditions (like zero packet loss). BTW, I also wish to make it possible to avoid relying on specific stream IDs for special functions. I hadn't considered opening order as a way to do that though and had assumed that we'd use an in-stream header for that. I don't think that makes the complexity worse, or at least no worse than having to deal with different frame types for different functions. On Tue, Apr 3, 2018 at 5:10 PM, Marten Seemann <martenseemann@gmail.com> wrote: > Hello, > > Sure, the application could deal with it by starting every stream with some > kind of header, to signal what kind of stream type this is. I'm not sure if > I like this solution, since it creates additional complexity, and it is only > necessary because we removed a useful feature from the spec. > What did we actually gain from removing implicit stream opening from the > protocol? As far as I can see, we were able to relax the requirement > "endpoints MUST open the send side of streams for each type in order" to > "endpoints SHOULD open the send side of streams for each type in order". > This might seem nice from a conceptual viewpoint, but honestly, I don't > really see why anyone would actually want to do this. I think the main > argument against the MUST was that it's not enforceable anyway, but this > applies to a lot of other MUSTs in the spec as well (e.g. that flow control > limits can't be decreased). > > The fix for this is straightforward: We should REQUIRE a peer to open > streams in order. How the receiver handles out of order streams then is an > implementation and API decision. I see two ways that an implementation could > reasonably deal with this: > > Return streams in the order they are received, e.g. return N+4 before N, if > packets are reordered. > Return streams ordered by stream ID, i.e. return first N and then N+4 if a > frame for N+4 is received. > > This way, we wouldn't need to reintroduce implicitly opened streams, but > leave this up to implementations. I should have named this thread > differently, if only I had realized this earlier ;) > > Regards, > Marten > > On Mon, Apr 2, 2018 at 7:05 PM Lucas Pardue <Lucas.Pardue@bbc.co.uk> wrote: >> >> Hi Marten, >> >> Would Stream headers fix this problem? I.e. anything that requires special >> behaviour other than "bulk data" has some bytes of of magic at the start of >> the stream. >> >> Regards >> Lucas >> ________________________________________ >> From: QUIC [quic-bounces@ietf.org] on behalf of Marten Seemann >> [martenseemann@gmail.com] >> Sent: 02 April 2018 12:43 >> To: QUIC WG >> Subject: Implicitly opened streams and exposing stream IDs >> >> Recently, the implicit opening of streams (i.e. that when a receiver >> receives a frame for stream N+4, it can assume that stream N was already >> opened, but the packet might have been reordered) was removed from the >> draft. I think this change has some consequences that we haven't discussed >> so far. >> >> For the purpose of writing some pseudocode, I'm assuming a QUIC API that >> provides an AcceptStream() method, but the conclusions will be the same for >> a callback-based API. >> >> * If QUIC has implicit stream opening, AcceptStream() would return the >> streams in order (and if a frame opening stream N+4 is received before >> stream n is opened, AcceptStream() will first return stream N and then >> stream N+4). >> * Without implicit stream opening, AcceptStream() just returns >> whatever stream is received first. Streams might be returned in arbitrary >> order, if the peer doesn't open streams consecutively or if packets are >> reordered. >> >> Now imagine an application protocol where the first unidirectional stream >> opened by the client is a control stream, and all higher unidirectional >> streams are data streams. The application on the server side needs to find >> out which stream is the control stream, because it needs to be handled >> separately. >> >> With implicit stream opening, the server code would be: >> >> control_stream = AcceptStream() // is guaranteed to open the first >> stream >> // handle the control stream >> while true: >> stream = AcceptStream() >> // handle the data stream >> >> and without implicit stream opening: >> >> while true: >> stream = AcceptStream() >> if stream.ID() == kControlStreamID: >> // handle the control stream >> else: >> // handle the data stream >> >> In this case, after accepting a stream, we first have to check the stream >> ID, since there's no guarantee if the control stream will actually be >> received first. >> >> For this stream mapping, it seems like the removal of implicitly opened >> streams implies that QUIC has to expose stream IDs to the application layer. >> I'm not sure if this was intended when making the change, especially since >> we're considering to change HQ such that it doesn't rely on QUIC stream IDs >> any more. >> We only manage to avoid the problem described here in our HTTP mapping, >> because the HTTP control streams are unidirectional and the request streams >> are bidirectional, and can therefore be told apart by their directionality. >> However, as a general transport protocol, other applications built on top of >> QUIC will have to find some way to deal with it. >> >> >> ----------------------------- >> http://www.bbc.co.uk >> This e-mail (and any attachments) is confidential and >> may contain personal views which are not the views of the BBC unless >> specifically stated. >> If you have received it in >> error, please delete it from your system. >> Do not use, copy or disclose the >> information in any way nor act in reliance on it and notify the sender >> immediately. >> Please note that the BBC monitors e-mails >> sent or received. >> Further communication will signify your consent to >> this. >> -----------------------------
- Implicitly opened streams and exposing stream IDs Marten Seemann
- RE: Implicitly opened streams and exposing stream… Lucas Pardue
- RE: Implicitly opened streams and exposing stream… Mikkel Fahnøe Jørgensen
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- Re: Implicitly opened streams and exposing stream… Marten Seemann
- Re: Implicitly opened streams and exposing stream… Martin Thomson
- Re: Implicitly opened streams and exposing stream… Mikkel Fahnøe Jørgensen
- Re: Implicitly opened streams and exposing stream… Marten Seemann
- Re: Implicitly opened streams and exposing stream… Martin Thomson
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- RE: Implicitly opened streams and exposing stream… Nick Banks
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- RE: Implicitly opened streams and exposing stream… Nick Banks
- RE: Implicitly opened streams and exposing stream… Nick Banks
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov
- Re: Implicitly opened streams and exposing stream… Ian Swett
- RE: Implicitly opened streams and exposing stream… Lubashev, Igor
- Re: Implicitly opened streams and exposing stream… Marten Seemann
- RE: Implicitly opened streams and exposing stream… Lubashev, Igor
- RE: Implicitly opened streams and exposing stream… Lubashev, Igor
- Re: Implicitly opened streams and exposing stream… Dmitri Tikhonov