Re: Prefer-Push, a HTTP extension.

Evert Pot <me@evertpot.com> Sat, 24 November 2018 00:26 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8AE5D128C65 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 23 Nov 2018 16:26:01 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.999
X-Spam-Level:
X-Spam-Status: No, score=-2.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, MAILING_LIST_MULTI=-1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=evertpot.com header.b=TOkAv6vf; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=DiwqgSKs
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gkYZMk_0R2uI for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 23 Nov 2018 16:25:59 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [IPv6:2603:400a:ffff:804:801e:34:0:38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3FF101271FF for <httpbisa-archive-bis2Juki@lists.ietf.org>; Fri, 23 Nov 2018 16:25:59 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.89) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1gQLjX-0006WC-FH for ietf-http-wg-dist@listhub.w3.org; Sat, 24 Nov 2018 00:23:39 +0000
Resent-Date: Sat, 24 Nov 2018 00:23:39 +0000
Resent-Message-Id: <E1gQLjX-0006WC-FH@frink.w3.org>
Received: from mimas.w3.org ([2603:400a:ffff:804:801e:34:0:4f]) by frink.w3.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from <me@evertpot.com>) id 1gQLjU-0006VZ-2I for ietf-http-wg@listhub.w3.org; Sat, 24 Nov 2018 00:23:36 +0000
Received: from out4-smtp.messagingengine.com ([66.111.4.28]) by mimas.w3.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from <me@evertpot.com>) id 1gQLjR-0007lX-7O for ietf-http-wg@w3.org; Sat, 24 Nov 2018 00:23:35 +0000
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id 91CFF21AA2; Fri, 23 Nov 2018 19:23:12 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 23 Nov 2018 19:23:12 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=evertpot.com; h= subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=mesmtp; bh=S2VuaEwAqUUphiuqTPTAsCF7yqS4f2bqSZCmoicAIeQ=; b=TOkAv6vf9UPt Pp6TtmwCypm++fwBJ9NshVC+Sl/3O94n1/HoZymNq67nX7RKzspcLbm8vN7HYcK1 L/QBu7AI4Bwo2Jf6gQsB20q5lgnSOVAqi3rT2rNLHNBlsx3wLERe8D+lEI1Mo+TR wwMs1YsxLIKQhoa1ttJcD+1DAVueqm8=
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=S2VuaEwAqUUphiuqTPTAsCF7yqS4f2bqSZCmoicAI eQ=; b=DiwqgSKssrLFtve21d5OzNMUy7tLX+xmQerKOzHUk1fbzj/spgcoboKK4 oAdpI34hp70OL1IfzX8+Vjftwx5eaDFSKv3JHzP+9ZAaPEh1sfwZmD8FH3Lccy66 hkdEnDFFuEhq9bJ2GOFU5actws9nsrmfXsnLAMwHKjzNsdGnAaoiXWHHerj+aouo ZLuDd80PNfmwDxx7TRhVs/KnVERB1Hv1m02OKIhYYliwetAJAjotvtmlVque51SF DMjyulDXDxMGItfBw5JIDobtYOgdCkQ7UUHcg5yopOwlZBsC5itYE+O6TnYOT4jv RPHUsnN+P0GUVM1grPh/VkUbjVJ3g==
X-ME-Sender: <xms:75n4W5-KXKd6TeCmiE0w9XUZQnQ_hOB5WFiuVfGube7ozmSIG7pkSg>
X-ME-Proxy: <xmx:75n4W1gSaBN1zfEAkd-7oSV7XsHEV4tEBWsp-uBcaIblpZIqR02oCA> <xmx:75n4W1DFAVZRNYidqEeBCmwOpByicMrYqC0ViF9qE3wJP3nL3WIWxw> <xmx:75n4W0GXd47C2fiYVBWCy5F-zzw0BqzuphloMNRzWPkU4BCUbMEVOQ> <xmx:75n4W7_fzVM3vNf66JB9sbTd33uCsHJlGuTfIBc7VpiZecFWZDaybA> <xmx:75n4WybrW8x_0zJyrR7tNAtVxDucjXIjkYVejXIFfHw-gF-O-HPGmw> <xmx:8Jn4W7AMREPtzHdIIH5d5eWy0wJNSIE6cFhd3L0x0-sZQjRbjd_Nig>
Received: from [192.168.0.26] (cpe0c473da1c313-cm0c473da1c310.cpe.net.cable.rogers.com [99.230.206.66]) by mail.messagingengine.com (Postfix) with ESMTPA id 6CEEA102A0; Fri, 23 Nov 2018 19:23:11 -0500 (EST)
To: Andy Green <andy@warmcat.com>, ietf-http-wg@w3.org
References: <18d0e93b-67e1-cfed-5418-fd50594fc5f9@evertpot.com> <873c1217-b07b-12cc-4c53-b9f75ebf3bdb@warmcat.com>
From: Evert Pot <me@evertpot.com>
Openpgp: preference=signencrypt
Autocrypt: addr=me@evertpot.com; keydata= xsFNBFtJFSYBEADPmEBaJC5Ey79441MLntdIDOecV/Jvro+k0nPT4pnlxyJX5nDDN7NP2FcW Z+QyQJ5Ib1K2OP317EE1RZ0yQVXdlBcG4Hn5ggUJ21cq3HAvOAs3CNuJtTtTcQWa+mMxcie1 27qcsvu4HZOoaEWnZl7nkhXcyj6VoBCrjCpnHr8bMDdcvj2tf6gLhqL+P0WflVd/5i8Y/3t3 nyiU7kTt49+h5P2h40oLc8IyO1LMHYf8937k//zImnBxOW/0h0uWAXawv0FJAKV6BcKu+3z7 woO7niTmlOmwHz1bF9BywDZmWsPZU8Etmthej3SH01LB96hEexjygOjVVcEbZEPnQxoyg1PR 4FgkYj/JFp80I4bOI49ZrUcjdxzjRS6yIvr2WTdqpEHbRayiuAWxA8OIt2aFjb9rPahZTyUt bn9g8mWCkKUqoKMbMiEQvpB2pNsDF5A25Z62FkSwk96a0I2NXEF47Xf3wpvtrBDm5WuuADfX OfAGsFdTU0X52uRlbfOnO+yDGmJnReWqewf95I7ikygbegNIQh8P7NSKK5mdCE8o7DiUb3iD rriBrp6qQmzvF1TezLjoI8MWDfAYWrRsxA4mwAKHIZ0HGLUZTA3bw9+07FRpL4oOdJUMc9J6 m8mP+HWE+gQpS7cinv9HC0FUp0Dhp/0BZkwvsslQQ9FdQCsMiwARAQABzRtFdmVydCBQb3Qg PG1lQGV2ZXJ0cG90LmNvbT7CwZQEEwEIAD4WIQSkMuXfRzs70V6UIiq3UVOR0jM1HQUCW0kV JgIbIwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRC3UVOR0jM1HdAgEACmQtD4 GCyhdJ7EZd0PHlkHrjaCDnE7YRIZDT8977GxBxyQYeCdh7QoMhpW/fyFxBmL8AAlv9VgB/Jq 9Mb5/UbQdb8ZeRQ8qub/bn7X3pRzSp9qHZzT67Vd+qGHdlUegoLV4/rvhZHrV81dayHAZ8rf Mn3U4CkKFyan+ZK8Psou5TIP7L4fXz97P/K296p+Qp9vRvjCBiX+cls3xlSgHdOgIbuJCjp9 yMxnLw8kk3KUtb1epmqFzNOr01GGWcyksKoCyc8TtZWgJuT7yswapHn3tjCTvcVAqZiVr+RJ gFQhyr8S8P4NwK3Sgk0Ogz++mVjpa/2Rh1XSESeiRLG895ofNaS8hmrfOrxnTuejQ/YyeojK 7luEFYa/0OqK1pS9Z0wI42pdemFELGg704wyHQDEkYDLfoFXi+PHZI6EX6LGnvnvKBic4nHi DpYjdqR5cbjyAhJdIZRENpvmBaiRR4ZTAZXnQEX2Zq6tFAboNJJ6G5feNWyDPScgHO+ZNP71 28nIsEkSum3ymyRdhMkbeIEZ5BRv/RPhxSyt/40YBi3YIacSkO508L6ALcUCUN/bYRj2pDkN h7nsH2E11SeeqUXGQuMjvTmJL8go2ndods/gL0E2HBo4oExKvmdCJY1FZaI50d8KjUZJLWxH 4QDWD3QFaKkVQIv/5dFpq40TZjtbFc7BTQRbSRUmARAAuRzGx8azFVYPwszmYutW6rOnWOno 8+EcGL6Pmoe5/2czxxjqofp4Gsy41jbyKsSqyVjBHGzY0yOzZc7fmNb4m6ef8jFteWhRECmI 4vZl1/9/gekvxDEDqrvKH6RbN944MdS5qovINBbomxq7ND/Dl524sylq+51nmJSW0MqazwqL wHW46LC7bur3F/jzGsv3o5qtZK0PUQi/HSH68CT6NnIbyMdrcgvjNKm3hb2/9h9MASd1xv58 tLeIt1ndcgocZVgwAqExj7iGFXbU0N24kig3bV4i3zJtUW/OSRr8YUJEG8blCnn4cJrGcqz/ YjvOdXEWzpOmQ+eVg7CPFO+gwdG4WaS5DdAcE6F/ooXQT+dgQ5hU4vgKmvso+ckd/0kuMhMH 5x8G91YjqgucEhBA1h4npy/KJVuDj8/qpbgVxtyoYTYuIgA3avK7lxZNb9ZxH+oqYFhkDjHg T56aBU0BAl1CcH7pddh9TY38Joj69cNoImXSL0xUc6qQxd+aFcT2dpFRVkNvfz9DA2/Q8gTA J3U1s9w2wdkZzK0saFzuvuPCAQjytNfn5hIuRyr871XUD9JV/uxbEiJBIBJl7sXpMsjupYKs m5cWo4wtVsDPgt2EmmiZR2hCo43BUhznX7vfeGos4tX4XIAyTr9y/KZA/y1Qq16bZqI1MiHL /ueJLI8AEQEAAcLBfAQYAQgAJhYhBKQy5d9HOzvRXpQiKrdRU5HSMzUdBQJbSRUmAhsMBQkJ ZgGAAAoJELdRU5HSMzUd3jYP/2iaMvJx9AUZBbfn/qidsd3an4sVeNb0Pn3webhxYhVvx4lV oFwfnQzQ9c4c+LMQ3QS6avYxLaRGQEDssCgHp+M4bhfchAbKfkDp0Fsk3XrqT3dqc41ljP+d n7Ov2qjS2fYjMet3APJw0fLmb9Y6Z4qd3SfVB3HblH0Lw+XgZJna6fEwJIb2F2yn/vihmBCx A86o1PeXZLHsc+kI3jY17xuTwd954K006W0u7/aqyo6oDCZGUdbBk1hvLYdprdaLD26xA527 uBMSAnOraVwM00wiVbT8ETr3yn5aTcVqcCIc5PydppTtowvtisvOQH2Xe8ygkjivBbDC2aMa ZHTtj8OBVCQHotv0Iw7+aEx+7qswCEkOiIYbtxy/K1wpFrm9VyWNXDimhjekiqDsO9CHAMtF FpbC7yH3063XdmGtHKow2J6xSPDxegCL6xKcYy8Huu4OqMxByjhMjFryG5/nCNd377VRy4S9 N9KG0VJAX4d5WE2qxXIiF1QX8mhddIuyzF8Uluil/G94+RFnO0+9Rl3J6iNK3z/AvQTpjpDD hpZTmkXbReG5q0gl175BFhKR0I7NeEOktZh5BjqGjRYnI7r6LkpS2jhPEpNI2YE43SqYNqkJ ecxvs9rmd//9lA2rzvtXzd/rvO2rqZl5dqzLlnOraaEDpTbOcVeMbtbyKzPA
Message-ID: <030508ae-c7f1-16a4-ef66-4c31c704e4a9@evertpot.com>
Date: Fri, 23 Nov 2018 19:23:10 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <873c1217-b07b-12cc-4c53-b9f75ebf3bdb@warmcat.com>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-W3C-Hub-Spam-Status: No, score=-3.1
X-W3C-Hub-Spam-Report: AWL=1.683, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1gQLjR-0007lX-7O 23d077045fa6caebb1876bf7ca5dd28a
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Prefer-Push, a HTTP extension.
Archived-At: <https://www.w3.org/mid/030508ae-c7f1-16a4-ef66-4c31c704e4a9@evertpot.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/36095
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

> 
> I have always been a bit puzzled by how PUSH is supposed to be
> beneficial when the server doesn't know what the client has locally
> cached.  Nowadays versioned scripts such as those from
> ajax.googleapis.com are typically told to be cached locally for one year[1]

I agree, but for the main case we're trying to solve (embedding vs
pushing), both have this issue. This draft isn't intended to solve that
problem, but the hope is that once Cache Digest[1] lands this _does_
become a viable optimization. (even more so if ETag makes its way back
to key calculations).

> 
> In the case "self" serves everything and all the assets have similar
> caching policies, after the first visit any PUSH stuff riding on dynamic
> HTML is going to be 99.99% wasted.
> 
> The draft doesn't seem to address:
> 
>  - why would this be beneficial compared to just sending n pipelined
> GETs on h2, if the client understands it wants n things already?  Both
> ways the return data has to be serialized into individual streams with
> their own headers on a single h2 connection.  With HPACK and n GETs that
> differ only in the request URL, the header sets for each request are
> cheap and you don't have to worry about either magicking up a new format
> to carry the info or "market penetration" of implementation

One major benefit is if a server knows in advance that a client will
want certain resources, it can optimize for them.

I hope my pseudo-code example illustrates this, but here's the general idea:

function controller(request, response) {

  if (request.preferPush()) {

    response.push( allChildResources() );

  }

}

Generally it's a lot cheaper to generate responses for a group of
resources (based on for example a SELECT query), than it is to generate
responses for each individually.

Plus, for all these pushed responses a service might have done a bunch
of work that doesn't need to be repeated for every response. Consider
that the original request had authentication information, the server
doesn't need to check Authorization headers for every request.

> 
> The draft says with its method "it's possible for services to push
> subordinate resources as soon as possible" but it doesn't compare it to
> just doing n GETs from the start.  I think you find any advantage is
> hard to measure.  But at least the draft should fairly compare itself to
> the obvious existing way to do

A client can only know which links are available and where they point
to, after the initial response came back. After that, I agree, a client
can just do GET requests for every linked resource individually and get
the same performance (not considering the fact that servers can optimize
for groups of similar requests).


>  - where does the contemporary knowledge come from at the client about
> the relationships?  From the server, ultimately?  Then this is a bold
> claim...

The biggest use-case from my perspective is for hypermedia-style API's,
such as HAL & Siren. In these cases clients generally do have knowledge
of which links might potentially be available, but not where they will
be pointing to.

Solving this for HTTP-services that don't follow this paradigm is out of
scope for this (for me at least).

> 
>> It reduces the number of roundtrips. A client can make a single HTTP
> request and get many responses.
> 
> h2 pipelining doesn't work like h1 pipelining.  You can spam the server
> with requests on new streams and most (all?) servers will start to
> process them in parallel while serving of earlier streams is ongoing.
> The server cannot defer at least reading about the new stream starts on
> the network connection because it must not delay hearing about tx credit
> updates or it will deadlock.  So there is a strong reason for servers to
> not delay new stream processing.

Yes, sorry, this is just to avoid having to wait for the first response
(or subsequent responses in case a bigger part of the graph is
requested), I don't expect it to optimize the case where a client
already knows the target of the links.

Anyway, point taken though. I think the draft needs to do a much better
job addressing this. I also think we need to get more real-world data.

Evert


> 
> -Andy
> 
> [1] "The CDN's files are served with CORS and Timing-Allow headers and
> allowed to be cached for 1 year."
> 
> https://developers.google.com/speed/libraries/


[1]: https://tools.ietf.org/html/draft-ietf-httpbis-cache-digest-05