Re: Deadlocking in the transport
Dmitri Tikhonov <dtikhonov@litespeedtech.com> Wed, 10 January 2018 22:36 UTC
Return-Path: <dtikhonov@litespeedtech.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6C270127599 for <quic@ietfa.amsl.com>; Wed, 10 Jan 2018 14:36:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level:
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=litespeedtech-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8tQEO9n8df5S for <quic@ietfa.amsl.com>; Wed, 10 Jan 2018 14:36:46 -0800 (PST)
Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1922E127369 for <quic@ietf.org>; Wed, 10 Jan 2018 14:36:46 -0800 (PST)
Received: by mail-qk0-x233.google.com with SMTP id q1so1531381qkb.9 for <quic@ietf.org>; Wed, 10 Jan 2018 14:36:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=litespeedtech-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to:user-agent; bh=rye00CwlF86J9tl/wRNrHv768EYBONTLKjBlMGiIAMY=; b=PLQK/f18PwWH/+hO8bmF8BwJ8q8F7P/9pE0S6kC4Qyt2C0HCtxoVUt7iJn4DMgxPuO RX1fmJTy/H2maATe4OhYFSLtY1RjycfN0PM4iegkW/VGCN46xAUrSArPw0rYm7tw4ydA fmx8VYkbhuOlEUGkHvd6HKvIAojqmi0fCGds28T+MLW1KufdIWp+dhwgthUXsOlf3GP0 zJk3ajjPSys5xSk7yZDJot/OuS/90eZJMd1GkTECPZhiM8TGDLEt2/kf8VC5t6bn1zmP XObu/mdQzhHG7YQ1onRoE21juX4y7+yaTmZIMln4CM45SzzFxGmj9/8nOFyPyGYLZMZ3 g79A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=rye00CwlF86J9tl/wRNrHv768EYBONTLKjBlMGiIAMY=; b=GC9I76OoCoyYVWuxmSmcxPEnKuYSM1jUew3mUiVS18Y8VjoJtuO81a0OW+qm761EyS eQFelSVriLL5+E8j0fbZQxQzjxCE0duVt2+b8xRxp0UL5Qv0XYBGtHN9SItU5AeLr/y1 4vepyliuKfYby4uYGLtyJ0gAsdUXBHSuH7A2ODDY+mMRE9rEHa6aRFTf5pOIhDq+pPFg CbidYNpUu1TMtCxYSEftkbKVZz2EFlSja/lCQ0R7nc4rfIuIy2tLLpcrArjsr2cibsGB Z0SDsL2AgUyjo7BlH6f1+EuQ2Dv1LAQP9MlqiuVbINgZ+leIymauyuEKKxosaLrEHkl4 tU0A==
X-Gm-Message-State: AKwxytdb623frZeHRe9PJvEYfevMPtMd27ysAgoDp1m7fbHsvk8JKM6E zNtrFP/4c7MlhR6T3JQkbf0OUg==
X-Google-Smtp-Source: ACJfBotaSyK2zTAOQcTZlZKFzOwsqnN6/SBsAheSsJxwGENBOOWLGUfZevuIbizElIR7WMv4nexyMw==
X-Received: by 10.55.88.7 with SMTP id m7mr10720937qkb.142.1515623804787; Wed, 10 Jan 2018 14:36:44 -0800 (PST)
Received: from ubuntu-dmitri (ool-2f1636b6.static.optonline.net. [47.22.54.182]) by smtp.gmail.com with ESMTPSA id y191sm10397083qky.63.2018.01.10.14.36.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 10 Jan 2018 14:36:44 -0800 (PST)
Date: Wed, 10 Jan 2018 17:36:42 -0500
From: Dmitri Tikhonov <dtikhonov@litespeedtech.com>
To: Martin Thomson <martin.thomson@gmail.com>
Cc: "Brian Trammell (IETF)" <ietf@trammell.ch>, QUIC WG <quic@ietf.org>
Subject: Re: Deadlocking in the transport
Message-ID: <20180110223642.GC7007@ubuntu-dmitri>
Mail-Followup-To: Martin Thomson <martin.thomson@gmail.com>, "Brian Trammell (IETF)" <ietf@trammell.ch>, QUIC WG <quic@ietf.org>
References: <CABkgnnUSMYRvYNUwzuJk4TQ28qb-sEHmgXhxpjKOBON43_rWCg@mail.gmail.com> <65EBA092-9DEF-4C9E-9DB8-AA490CA60598@trammell.ch> <CABkgnnVFHu6UhdUJYLUKXJyz7oGVdqKpaYz=eEhUR5_nNOQeXw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CABkgnnVFHu6UhdUJYLUKXJyz7oGVdqKpaYz=eEhUR5_nNOQeXw@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/UgapfdFsOQuNIPB7q5jUgHaA9AU>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 10 Jan 2018 22:36:48 -0000
On Thu, Jan 11, 2018 at 08:54:03AM +1100, Martin Thomson wrote: > On Wed, Jan 10, 2018 at 6:04 PM, Brian Trammell (IETF) <ietf@trammell.ch> wrote: > > Do we have any actual numbers on how much efficiency can actually be > > squeezed out of allowing compression dictionaries to create unbounded > > dependencies between streams on common, non-pathological workloads? > > Dmitri ran some tests with QMIN, which doesn't create dependencies, > and found that for a time compression efficiency degraded noticeably > if it couldn't create dependencies. I'll let Dmitri speak to that > point. The shortcoming rectifies itself reasonably quickly, though What we have is a set of simulation results comparing QMIN (conservative: no blocking) with QCRAM and QPACK (optimistic: blocking allowed). I used two web session traces: one visiting www.netbsd.org (small trace) and another visiting www.facebook.com (larger trace). I will paste parts of two reports: 1> I integrated my QMIN library [1] into Alan's proxygen simulator [2] to 1> assess QMIN's compression performance. Specifically, we want to see how 1> QMIN performs a) in a short-lived connection and b) in a longer connection. 1> I generated two HAR files: 1> 1> - I typed in www.netbsd.org into the Location bar of my browser and 1> waited for the page to load. This resulted in 18 requests, 17 of 1> them to www.netbsd.org and 1 to www.paypalobjects.com. 1> 1> - I signed in to Facebook and clicked about a dozen links. This resulted 1> in 383 requests: about 100 to www.facebook.com, about 280 to fbcdn.net, 1> and a few odd ones like facebook.net and fbsbx.com. 1> 1> I left all loss-related simulator options unset for the initial test, as we 1> just want to look at the ideal environment compression performance first. 1> The dynamic table capacity is set to 4 KB. The results are as follows: 1> 1> (The simulator implements the latest QPACK and QMIN drafts; its QCRAM 1> implementation dates to June -- but it is better than nothing.) 1> 1> Table 1: Compression performance comparison, no packet loss 1> 1> HPACK QPACK QCRAM QMIN 1> 1> netbsd 1265/6555 1309/6555 1361/6555 2202/6555 1> Ratio: 80 Ratio: 80 Ratio: 79 Ratio: 66 1> 1> facebook, 58528/243576 61753/243576 61386/243576 63782/243576 1> blended [a] Ratio: 75 Ratio: 74 Ratio: 74 Ratio: 73 1> 1> facebook, 59325/243576 62594/243576 63462/243576 67492/243576 1> unblended Ratio: 75 Ratio: 74 Ratio: 73 Ratio: 72 1> 1> a: "blended" means that the simulator pretends that facebook.com and 1> fbcdn.net is the same domain name. This is true by default; to 1> disable, pass -blend=false on the command line. The unblended 1> simulation is closer to reality. 1> 1> How to read this table: Each entry is a compressed version of the last 1> three lines of the simulator output: 1> Uncompressed Bytes: 243576 67492/243576 1> Compressed Bytes: 67492 ====> Ratio: 72 1> Compression Ratio: 72 The second report addresses compression performance during the first few flights: 2> What I do have, however, is my short netbsd session (a HAR of which I 2> emailed to the group in an earlier email). This is as good a starting 2> point as any for drilling down into the compression performance of the 2> first few header blocks. The netbsd trace contains 17 requests to 2> www.netbsd.org; they belong to three distinct flights: 2> 2> 1. Request index.html (1 request) 2> 2. Request CSS and JS (3 requests) 2> 3. Request images (13 requests) 2> 2> It would be informative to examine compression performance in each of 2> the flights. 2> 2> Since the meeting of Dec 4, Alan added a cumulative compression ratio 2> reporting to the proxygen simulator [2]. In addition, I added a 2> per-header block compression ratio reporting to the QMIN encoder library. 2> First, compression performance comparison: 2> 2> Table 1: Cumulative compression performance comparison 2> 2> Req # QPACK QCRAM QMIN 2> 2> 1 241/39 225/42 303/27 2> 2> 2 78/58 74/60 219/35 2> 3 46/67 44/68 195/39 2> 4 47/72 52/72 188/42 2> 2> 5 52/75 52/75 110/49 2> 6 55/77 55/76 96/53 2> 7 55/78 55/78 96/57 2> 8 54/79 54/78 95/59 2> 9 50/80 50/79 91/61 2> 10 55/80 55/80 96/63 2> 11 56/81 56/80 97/64 2> 12 63/81 63/80 104/65 2> 13 52/81 52/81 93/66 2> 14 51/82 51/81 92/67 2> 15 47/82 47/81 88/67 2> 16 46/82 46/82 87/68 2> 17 79/82 77/81 94/69 2> 2> How to read this table: the two numbers separated by the slash are the size 2> of the encoded header block followed by the cumulative compression ratio. 2> These values are parsed from the simulator output: 2> 2> Encoded request=0 for host=www.netbsd.org block size=225 cumulative \ 2> compression ratio=42 2> 2> translates to 2> 2> 225/42 2> 2> (Note that the cumulative compression of all three does not match previously 2> reported results: this is because I do not take the last request into account 2> here, as it is going to a different domain.) Caveat: QCRAM and QPACK drafts have changed significantly since then. QMIN performance can be improved slightly while still keeping its non-blocking behavior. > Roberto makes the point that this is precisely the point where > compression matters most. It's a point that the header compression > folks are still debating. The opposite view is that packet loss in the beginning would have a worse impact than compression inefficiency. I am sorry to say that we do not have good simulation data to go on. > (The dependencies wouldn't be unbounded. One advantage of the > mechanism that QMIN proposes is that these dependencies are pretty > narrowly constrained and well understood.) Yes. That is, QMIN does not introduce any stream dependencies. - Dmitri.
- Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Brian Trammell (IETF)
- Re: Deadlocking in the transport Willy Tarreau
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Subodh Iyengar
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Ian Swett
- Re: Deadlocking in the transport Mikkel Fahnøe Jørgensen
- Re: Deadlocking in the transport Mikkel Fahnøe Jørgensen
- Re: Deadlocking in the transport Charles 'Buck' Krasic
- Re: Deadlocking in the transport Ted Hardie
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Roberto Peon
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Mikkel Fahnøe Jørgensen
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Christian Huitema
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Jana Iyengar
- RE: Deadlocking in the transport Mike Bishop
- Re: Deadlocking in the transport Mikkel Fahnøe Jørgensen
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Mikkel Fahnøe Jørgensen
- Re: Deadlocking in the transport Martin Thomson
- RE: Deadlocking in the transport Lubashev, Igor
- Re: Deadlocking in the transport Dmitri Tikhonov
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Roberto Peon
- RE: Deadlocking in the transport Lubashev, Igor
- Re: Deadlocking in the transport Mirja Kühlewind
- Re: Deadlocking in the transport Roberto Peon
- Re: Deadlocking in the transport Roberto Peon
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Jana Iyengar
- Re: Deadlocking in the transport Ian Swett
- Re: Deadlocking in the transport Martin Thomson
- Re: Deadlocking in the transport Mirja Kühlewind
- Re: Deadlocking in the transport Charles 'Buck' Krasic
- Re: Deadlocking in the transport Roberto Peon
- Re: Deadlocking in the transport Martin Thomson