[hybi] permessage-deflate in Chromium/pywebsocket and zlib's limitation regarding window size

Takeshi Yoshino <tyoshino@google.com> Thu, 16 February 2017 09:03 UTC

Return-Path: <tyoshino@google.com>
X-Original-To: hybi@ietfa.amsl.com
Delivered-To: hybi@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C5B691299DC for <hybi@ietfa.amsl.com>; Thu, 16 Feb 2017 01:03:15 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.701
X-Spam-Level:
X-Spam-Status: No, score=-2.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=google.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PmZ_HxSc39oE for <hybi@ietfa.amsl.com>; Thu, 16 Feb 2017 01:03:14 -0800 (PST)
Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4C1A01299D1 for <hybi@ietf.org>; Thu, 16 Feb 2017 01:03:14 -0800 (PST)
Received: by mail-wm0-x22f.google.com with SMTP id v77so9826049wmv.0 for <hybi@ietf.org>; Thu, 16 Feb 2017 01:03:14 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=HiYI7fROUA2gcKEMKisq+ONFUeNoseywmcvyPtBIQYI=; b=rHLdBvM7TxquZAusV71pETxPJPo9sQdaB4c04eb0Kuk/e74UYwVft3ACXwrPXCdTvi 60lOeSJ0d0OVmt5g0tGtKazZTQbHiT/ce/6TT99RV/Oi3vUOHp58a7pPo7yr3nj87Y6Y f4t00OZexinQVmw0Cw9259+X9sNKij12E1tCVv+aAoEMQzeW8BvBxdMFOvynFL8BVFda mPnugDpnu2yObwgElb6ZM6OypxrseuW7ron8cFpKuVZB87XdwlGkCPbq/0d1rRg4UAnH 8Rf1Bv73O/F/5GxmJfPACu6vv8Klc0dLio10q3vyBbvxkp7tWIGC5qLk3U2CbNUIWVfu jDLw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=HiYI7fROUA2gcKEMKisq+ONFUeNoseywmcvyPtBIQYI=; b=GiiiJYv8jkiMP6vyqnBEbrfeEEVmZodL7hrEN68dHaUlcYZSywTz3HCI24r27J+lNO H6rcUWSBNskDNk31BP1Z5Z4Rxcr353v6AivAhuO90iVbjgHDLQYUSI3DxzyyOnkQsLrN 3TkporrlPEmQIcr16xnk1d4XxjS8todZtlwV/h4n6j0WdAvvO925oFgb5TXDoBiI5EMm 3XQcpxiHuaEjXfEyc5N29m484EKhGOlmaSrN128DSHM5Wu8h8KSGFRpWWZgJKCCuW4ow +4T2QQ1Nv2vw+9LK7EhQypoBiM3SAVe5wDv54zABvsxKh4+36dSwQOejSSACA0tANmAR a3rg==
X-Gm-Message-State: AMke39klXM0p9HUhV/Aeme89uH8TQuvX58MEP3sldcQwKnY8/lebgnKCkJeAmJRYBD7fuWL+vXnMthK+f5pUhgWK
X-Received: by 10.28.191.79 with SMTP id p76mr11337360wmf.21.1487235792395; Thu, 16 Feb 2017 01:03:12 -0800 (PST)
MIME-Version: 1.0
Received: by 10.223.176.130 with HTTP; Thu, 16 Feb 2017 01:02:51 -0800 (PST)
From: Takeshi Yoshino <tyoshino@google.com>
Date: Thu, 16 Feb 2017 18:02:51 +0900
Message-ID: <CAH9hSJb8KtC5GTN9KZ-9qUjiLY2SgLG+K34AN9vcs2LhK1eDZQ@mail.gmail.com>
To: "hybi@ietf.org" <hybi@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c0712721ac4610548a211a2"
Archived-At: <https://mailarchive.ietf.org/arch/msg/hybi/lg5EK51jV4vlvFPnqTT_UAcjBV0>
Subject: [hybi] permessage-deflate in Chromium/pywebsocket and zlib's limitation regarding window size
X-BeenThere: hybi@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Server-Initiated HTTP <hybi.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/hybi>, <mailto:hybi-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/hybi/>
List-Post: <mailto:hybi@ietf.org>
List-Help: <mailto:hybi-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/hybi>, <mailto:hybi-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2017 09:03:16 -0000

Sending heads up to those who implement or use the permessage-deflate
extension as it might be useful.

zlib (and therefore Chromium and pywebsocket) has been behaving as if
window_bits parameter was set to 9 to compress data even when given the
window_bits parameter set to 8. See
https://github.com/madler/zlib/issues/171 for more details.

Recently, zlib has been updated to return an error when told to use bits 8
by
https://github.com/madler/zlib/commit/049578f0a1849f502834167e233f4c1d52ddcbcc

Chromium hit this error when rolling the new zlib. We decided to explicitly
tell zlib to use window_bits 9 when told to (e.g. by client_max_window_bits
from a server) use 8 to keep the existing behavior. See
http://crbug.com/691074 for more details. For the mean time, we'll just
keep this behavior.

Q: Any visible change made on Chromium's behavior in http://crbug.com/691074
?
A: No

Q: Chromium has not been and is still not honoring client_max_window_bits
correctly?
A: Yes, but effectively no, fortunately ... Because of "Compression Factor
Design Quirk" (see http://www.zlib.net/zlib_tech.html), Chromium has been
using 250 byte back-reference which can be decompressed by an inflater with
256 byte long back reference.


Thanks mark, ricea, yhirano for analysis at http://crbug.com/691074.