Re: Broader discussion - limit dictionary encoding to one compression algorithm?
"Roy T. Fielding" <fielding@gbiv.com> Tue, 21 May 2024 23:37 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=ietf.org@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BA18DC1840E3 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 21 May 2024 16:37:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.848
X-Spam-Level:
X-Spam-Status: No, score=-2.848 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, MAILING_LIST_MULTI=-1, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=w3.org header.b="hEsE1X1J"; dkim=pass (2048-bit key) header.d=w3.org header.b="X+4Qyjjj"; dkim=pass (2048-bit key) header.d=gbiv.com header.b="otQpAxlW"
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oUFQo0zN7Dr1 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 21 May 2024 16:37:14 -0700 (PDT)
Received: from mab.w3.org (mab.w3.org [IPv6:2600:1f18:7d7a:2700:d091:4b25:8566:8113]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C16D3C16943F for <httpbisa-archive-bis2Juki@ietf.org>; Tue, 21 May 2024 16:37:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=w3.org; s=s1; h=Subject:To:References:Message-Id:Cc:Date:In-Reply-To:From: Mime-Version:Content-Type:Reply-To; bh=hzUKz2JLMxJs9Egq0NVpFO+TrZNyykhhix70ZOOT0e0=; b=hEsE1X1JF6KGwju+4hU850WM2H KmsVPWc8Py6wSgdHP3yWQBH5kp+LfbYHFOlXBdz1R3UvD2DiidMs5I8qhL4bVMLHYhStD5q+pwZb5 GJqeCJusSMTfHBb8jFIrk2G+9R1WUQES914tG4ffmqfsy8w8givimeccM4lAUqgvaBsS/mJg6ebCO tySUbgRd4k2KKLCbrgnjE2ywvO9uBh16MRDXwgEep/2e7+A6NQfD4W48EP9J+u4OsdxLxTtDBR904 5lg21D7rLeNHyZipXdc1FYOu2wyhtXoZZY0VK+NO/tzPDq9h915LAfpVi5HYUWE3HVQT5M8Inq/wK k9oxKu8Q==;
Received: from lists by mab.w3.org with local (Exim 4.96) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1s9Z1x-00GHni-36 for ietf-http-wg-dist@listhub.w3.org; Tue, 21 May 2024 23:36:29 +0000
Resent-Date: Tue, 21 May 2024 23:36:29 +0000
Resent-Message-Id: <E1s9Z1x-00GHni-36@mab.w3.org>
Received: from ip-10-0-0-224.ec2.internal ([10.0.0.224] helo=puck.w3.org) by mab.w3.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from <fielding@gbiv.com>) id 1s9Z1v-00GHmj-2R for ietf-http-wg@listhub.w3.internal; Tue, 21 May 2024 23:36:27 +0000
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=w3.org; s=s1; h=To:References:Message-Id:Cc:Date:In-Reply-To:From:Subject: Mime-Version:Content-Type:Reply-To; bh=hzUKz2JLMxJs9Egq0NVpFO+TrZNyykhhix70ZOOT0e0=; t=1716334587; x=1717198587; b=X+4QyjjjqpgfxVUscpy5DuIVzDixNe7u+0Q6pLHBsm84LnfhBVseI6Y3gaDGiZElWy8SNO7Rjo7 32bGQmg9N6RwiU5H6b9xY6zbShpQw/saMF6N+cBMDgek8kLGwVdpxTyZo5EwEXFiAwUSyIJSuX+4j iztXNthlz1/qALWEcihoGSArbf5b++LxC97ArDXFhUevshP5IndQbB1aWQMQdOVspvb/RuB7lG+SI w5lH49aXT2qTS22pphhPYYeQjWoL6NLi+EtKaXHXJl4b3hCT60Crh3w8Wfj1o92X50DSqQBVDPtd+ i0GiXQebRsb+CL51Bm6KVayDNTE8rJGJzbCQ==;
Received-SPF: pass (puck.w3.org: domain of gbiv.com designates 23.83.218.249 as permitted sender) client-ip=23.83.218.249; envelope-from=fielding@gbiv.com; helo=poodle.tulip.relay.mailchannels.net;
Received: from poodle.tulip.relay.mailchannels.net ([23.83.218.249]) by puck.w3.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from <fielding@gbiv.com>) id 1s9Z1v-00HCkl-08 for ietf-http-wg@w3.org; Tue, 21 May 2024 23:36:27 +0000
X-Sender-Id: dreamhost|x-authsender|fielding@gbiv.com
Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id C5934C0E04; Tue, 21 May 2024 23:36:22 +0000 (UTC)
Received: from pdx1-sub0-mail-a231.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 59E82C0CF1; Tue, 21 May 2024 23:36:22 +0000 (UTC)
ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1716334582; a=rsa-sha256; cv=none; b=vhRuvAVSRe1LXpvZo0GkUCJPn78DKDDT66CDqJkw648v7Ea5GQEvu5SVbmZImnvj4gS7V/ /6udm9uyvs3nd0XXFPnf7YUMHibrNsJ0X7enQER9UwkpMV17pfoLlZ4jcBmr2UIsTC5GBk qW7NORbCwSxup9dm1hEeHdWwqeEfFjppV5odyaPeAl1tQX8SkicCYnBIpbl/hV8R2nBDId 57do2VMYdEBreDrRy9fAxrH/MykNxpk9HRiR7FrlklOkeTI/02dADbVSNz9hMTAySKvrmn 6g420XycyRde+z5VvTo4IJhhOdUX2O+YWesLLwue7ELpFn5ll6N5DShf0ck86g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1716334582; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wdoas8fIykLxjWR7XNJTOIb7nnnhhzslvSpmE7W1ETQ=; b=J2PVy1PZ9s/XErfLtq0Tp7xWRJtyzpFsIdY+ptgv0WyaxMQKBK9ziZpLGxh6VQS63cf/JA PWirF8m2ELimKhWVfn2gT41TKegLuwrI9jFhxo1BYlUZYnli+6I+YSLX8gES0q010l+fh3 bjmNSnMJONTdtka02nLHtT25wCTjPtCoxqAbW1PBhzXARdHtbHmDawTmJpB87PnH1w3dtw PL5VvV/xARJwJEXpShFvf0nmHbjLXV4fdr3Od7cUVt/QJWm7Uf4UPg6EaLjFJTogFXdmOl wXY2mtyKMQGPgDZuRs2BYA0gesno6oOqMCRn4RubuWtk7WpkTn+1wlK/IlaGVQ==
ARC-Authentication-Results: i=1; rspamd-68bbddc7f5-2sxfq; auth=pass smtp.auth=dreamhost smtp.mailfrom=fielding@gbiv.com
X-Sender-Id: dreamhost|x-authsender|fielding@gbiv.com
X-MC-Relay: Neutral
X-MailChannels-SenderId: dreamhost|x-authsender|fielding@gbiv.com
X-MailChannels-Auth-Id: dreamhost
X-Versed-Stretch: 3b4cd49f030b1ae5_1716334582644_3423634763
X-MC-Loop-Signature: 1716334582644:875435931
X-MC-Ingress-Time: 1716334582644
Received: from pdx1-sub0-mail-a231.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.102.166.185 (trex/6.9.2); Tue, 21 May 2024 23:36:22 +0000
Received: from smtpclient.apple (ip72-194-73-53.oc.oc.cox.net [72.194.73.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: fielding@gbiv.com) by pdx1-sub0-mail-a231.dreamhost.com (Postfix) with ESMTPSA id 4VkW6G08YXz1q; Tue, 21 May 2024 16:36:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gbiv.com; s=dreamhost; t=1716334582; bh=hzUKz2JLMxJs9Egq0NVpFO+TrZNyykhhix70ZOOT0e0=; h=Content-Type:Subject:From:Date:Cc:Content-Transfer-Encoding:To; b=otQpAxlWg8XBHqFiMTFY1YO3e7SYAcg7qE51aWtJHVja3EIHqjOXNK93gjO2j/apg q+4edJ5LeQbiMA2Pf4HjwuWCR44gywGM9hMtcJRau2oyrzvVylrDnmxXDgXHqTVTaj jccHVDk4iScIdlEN+WQNCnzRhvroQn0U41Tr56ijz4UpIYBHN4P+ERQxGuBXKWyRdS axPZf2jCpWEVeLpWZ9c6FqOFFh+mUTLHbNzxysBYbgCDxbqhZxdZPCneiScK14skiZ TCuIBg4WfMbO99JHU67exszsp0aKi9zi3IbJkNXjoISj2YQvkfg2VbYupU6kaWa8Hf mt+foq6Sg89SA==
Content-Type: text/plain; charset="utf-8"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\))
From: "Roy T. Fielding" <fielding@gbiv.com>
In-Reply-To: <ZkzhiRlXfG30_FVx@xps13>
Date: Tue, 21 May 2024 16:36:10 -0700
Cc: Patrick Meenan <patmeenan@gmail.com>, HTTP Working Group <ietf-http-wg@w3.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <9FB45A86-1E17-4C15-A047-7E59A49E93D3@gbiv.com>
References: <CAJV+MGzjUnZZ=XFn5veOvuhVWyZNP2b9U0fxpS3UmrDC_bc_wQ@mail.gmail.com> <202405211641.44LGfY2U006906@critter.freebsd.dk> <CAJV+MGzwbFAC7NhP611HDaVYMjMX0Q+KZ-QYirFu5WzjWL771g@mail.gmail.com> <ZkzhiRlXfG30_FVx@xps13>
To: Glenn Strauss <gs-lists-ietf-http-wg@gluelogic.com>
X-Mailer: Apple Mail (2.3774.600.62)
X-W3C-Hub-DKIM-Status: validation passed: (address=fielding@gbiv.com domain=gbiv.com), signature is good
X-W3C-Hub-Spam-Status: No, score=-6.1
X-W3C-Hub-Spam-Report: ARC_SIGNED=0.001, ARC_VALID=0.001, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, DMARC_MISSING=0.001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_WL=-1
X-W3C-Scan-Sig: puck.w3.org 1s9Z1v-00HCkl-08 d363d16d1262bcdba4491a41f6e4739a
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Broader discussion - limit dictionary encoding to one compression algorithm?
Archived-At: <https://www.w3.org/mid/9FB45A86-1E17-4C15-A047-7E59A49E93D3@gbiv.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/51957
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/email/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
> On May 21, 2024, at 11:01 AM, Glenn Strauss <gs-lists-ietf-http-wg@gluelogic.com> wrote: > > On Tue, May 21, 2024 at 01:02:30PM -0400, Patrick Meenan wrote: >> On Tue, May 21, 2024 at 12:41 PM Poul-Henning Kamp <phk@phk.freebsd.dk> >> wrote: >> >>> Patrick Meenan writes: >>> >>>> ** The case for a single content-encoding: >>>> […] >>>> ** The case for both Brotli and Zstandard: >>> >>> First, those are not really the two choices before us. >>> >>> Option one is: Pick one single algorithm >>> >>> Option two is: Add a negotiation mechanism and seed a new IANA registry >>> with those two algorithms >>> >>> As far as I can tell, there are no credible data which shows any >>> performance difference between the two, and no of reason to think that any >>> future compression algorithm will do significantly better. >>> >> >> We already have a negotiation mechanism. It uses "Accept-Encoding" and >> "Content-Encoding" and the existing registry. Nothing about the negotiation >> changes if we use one, two or more. The question is if we specify and >> register the "dcb" content-encoding as well as the "dcz" content encoding >> as part of this draft or if we only register one (or if we also add a >> restriction that no other content encodings can use the dictionary >> negotiation). >> >> As far as future encodings, we don't know if any algorithms will do better >> but there is the potential for content-aware delta encodings to do better >> (with things like reallocated addresses in WASM, etc). More likely, there >> will probably come a time where someone wants to delta-encode >> multi-gigabyte resources where the 50/128MB limitations laid out for "dcb" >> and "dcz" won't work and a "large window" variant may need to be specified >> (as a new content encoding). > > A practical approach is to allow for future unknowns as you describe > above, and to pick one of (brotli, zstd) to be required to be > implemented in this version of the standard, with the other optional. > Future versions might have a different required algorithm, and include > the intention that an algorithm required in a prior version will remain > an option in future versions. > > If a content-provider spends the time to build procedures and > infrastructure to deploy compression dictionaries, they will probably > experiment with their content and their CPU, memory, and other resource > limitations; and along with client capabilities, balance their choices > based on all of those inputs and more. > > Proxies and CDNs may make different choices on what they support > receiving from origin servers, how they store it, and what they support > sending to clients. Intermediate security scanners and possibly > alternate corporate malware will add other limitations. > > Compression dictionaries are an optional feature and one that an origin > server might choose not to implement (or a server deployed before and > older than the specification). I think the problem is that this choice is being characterized as a negotiation, as if the client is able to choose from multiple encodings, whereas in common practice the origin server chooses one encoding or none at all. IOW, the client's communication is advising the server on what will work for this response, not which one is best. It's not really a negotiation -- we just call it that in HTTP. The server will choose what is best based on their expectations. That includes deployment, type of client, type of resource, type of content, and likely changes to the representation over time (e.g., efficiency of the encoding). The chosen encoding may change over time, as new encodings are deployed or new things are learned about likely changes, and the choice may differ by anticipated client type. That's why we need labels for the chosen encoding. I don't think anyone can know that there is only one algorithm needed for dictionary-compression. I agree with the prior comment on window-size differences, particularly for brotli. What a browser wants in terms of reasonable window sizes is totally different from what a continuous-integration platform might want, or a software update platform might want. It's incredibly important to keep in mind that ALL of them are using HTTP. We cannot fall into the trap of thinking that communication over HTTP has to be tailored specifically for certain clients, like the few remaining general-purpose browsers. We cannot allow HTTP itself to be carved up into application-specific battles over syntax or named parameters, since its purpose is to be that one uniform interface which isn't application-specific. HTTP is extensible, by design. Also, I don't believe the two existing encodings are equivalent for all content types. I think brotli is better for some and zstd is better for others, largely depending on the data format and the nature of each resource's changes over time. If the current browser vendors want to deliberately choose only one encoding to implement, at least for now, that's fine with me. The IETF doesn't need to make that decision for them. I don't even need to know why such a choice is made, so long as it doesn't prevent other implementations from making their own choices, for their own reasons. We use self-descriptive protocols to enable extensibility, in terms of both our limited imagination of what can be done now and our limited ability to anticipate what will be needed long into the future. I see no difference here, and no need to pick a winner when the default is to not use it at all. ....Roy
- Broader discussion - limit dictionary encoding to… Patrick Meenan
- Re: Broader discussion - limit dictionary encodin… Poul-Henning Kamp
- Re: Broader discussion - limit dictionary encodin… Patrick Meenan
- Re: Broader discussion - limit dictionary encodin… Glenn Strauss
- Re: Broader discussion - limit dictionary encodin… Roy T. Fielding
- Re: Broader discussion - limit dictionary encodin… Jyrki Alakuijala
- Re: Broader discussion - limit dictionary encodin… Patrick Meenan
- Re: Broader discussion - limit dictionary encodin… Jyrki Alakuijala