RE: Server Push and Content Negotiation

Mike Bishop <Michael.Bishop@microsoft.com> Wed, 24 August 2016 18:46 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5ECFB12D5D1 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 24 Aug 2016 11:46:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.569
X-Spam-Level:
X-Spam-Status: No, score=-7.569 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.548, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tivpftbPG7-4 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 24 Aug 2016 11:46:02 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A025312D549 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 24 Aug 2016 11:46:02 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1bcd7F-0001h1-B2 for ietf-http-wg-dist@listhub.w3.org; Wed, 24 Aug 2016 18:41:33 +0000
Resent-Date: Wed, 24 Aug 2016 18:41:33 +0000
Resent-Message-Id: <E1bcd7F-0001h1-B2@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <Michael.Bishop@microsoft.com>) id 1bcd77-0001gG-C7 for ietf-http-wg@listhub.w3.org; Wed, 24 Aug 2016 18:41:25 +0000
Received: from mail-co1nam03on0096.outbound.protection.outlook.com ([104.47.40.96] helo=NAM03-CO1-obe.outbound.protection.outlook.com) by lisa.w3.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.80) (envelope-from <Michael.Bishop@microsoft.com>) id 1bcd74-0004ta-0r for ietf-http-wg@w3.org; Wed, 24 Aug 2016 18:41:24 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hndYSaigKNJd4AvmqhCVwDAkA8WhXzQyDwnRemgkfU0=; b=KRs6HFiNrvi7Hbz67u3N/5fzj2gN+6AuZxeV7X7GVEsClTcKUR09foc9UbH7GKkHiYb0TIaaVUxwEv3KJTn1Rwr+On/G/NjmN0k0wVk4vc40+gCh6xsK+YxzZtvv7/djeaevGXnpXeK7RzWTjpTqk66Cchgs9Ltt9CNalgGhw5o=
Received: from BY1PR03MB1338.namprd03.prod.outlook.com (10.162.109.20) by BY1PR03MB1339.namprd03.prod.outlook.com (10.162.109.21) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.587.9; Wed, 24 Aug 2016 18:40:51 +0000
Received: from BY1PR03MB1338.namprd03.prod.outlook.com ([10.162.109.20]) by BY1PR03MB1338.namprd03.prod.outlook.com ([10.162.109.20]) with mapi id 15.01.0587.013; Wed, 24 Aug 2016 18:40:52 +0000
From: Mike Bishop <Michael.Bishop@microsoft.com>
To: Mark Nottingham <mnot@mnot.net>, HTTP Working Group <ietf-http-wg@w3.org>
Thread-Topic: Server Push and Content Negotiation
Thread-Index: AQHR/cBY+JNisGH2aEOwCMQWDG3A5aBYcAMQ
Date: Wed, 24 Aug 2016 18:40:52 +0000
Message-ID: <BY1PR03MB1338B77F08376669853F92D487EA0@BY1PR03MB1338.namprd03.prod.outlook.com>
References: <F638033C-C1E0-447D-915E-A0B947629EA7@mnot.net>
In-Reply-To: <F638033C-C1E0-447D-915E-A0B947629EA7@mnot.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=Michael.Bishop@microsoft.com;
x-originating-ip: [2001:4898:80e8:9::390]
x-ms-office365-filtering-correlation-id: e93c9201-22cb-4010-d84a-08d3cc4e2d4e
x-microsoft-exchange-diagnostics: 1; BY1PR03MB1339; 6:Drbh9k7JnZXVotZ1OiHNR7dTpf12CCk7KXOvrsMEc5WI3S9ObpkQ3Gqt2qJGjSlwluMgEYQbKt9DGQbLpeCSrdwLuLPMnoo2fFVGX/kZ1ChOq1AMrw/NC/x9GvkBUEC/0n2o25ZpAHj7Yf3MI658fUEdSEzFQ+/Alld/2EN0GA0qgClYkBekFmL0SF40wrgUoLwTAMEwelWTxDEldy9rGlfz2hIsXufNt4HNbKnNhMVcGSw1rb1i2KjtE+41S7Gmg1+5DhYy0GdGIv7arxmo20UC6LlyFzW8P/6E1rmVUVCj8sTcIXJx8AtXfa1kgxxBCb4I4xMxitq9KPIbyF7AiQ==; 5:rdZyZOeVL+A2mYppsXuFVEhPt3w9F0UKrbH0hcIvnDMtc6UEjIvI8CoaSkdZqWGObqI4IBLzeefi+rkFoQJeWfJNMDTkiTjrCNCIWSH71Kwx6fyxkmt301JFltCtcN8PNNWEw/mHFDbW+4lWETaakA==; 24:won38KPIwIXQH+5fbpIyfJpeLwG+VBK2eAiUQBYZIi2JTaxVmtzXdKJe8LgJ0UUT29kMKVB9clzGVtRkfS/N4a3dI+WHBAYg2FRqPPaaHPw=; 7:Q6yJ/S6UT+tZB5lxut9Y+MTKYQslj7bZvOSCF+LT0H/+DEr4zybfalGXI0ImJcuZfA1B6q8BKw1V3aE8y6Lf0mqNyKktKtrp83n8g2yz7jtl6kMhKjOcbdqGCXYOqVF5IOsuBMWfWPfHKPR+cuGc+KA7Qr1pXqibqsNRPEwuN6rdxxBhAMWGtzw3uLqQMvYTAyDXx+DA9wXcar0mTQA4cJfSGaDzi2hctjsqGbFZtpe3L4M0e/oa137NMxMA9bPp
x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR03MB1339;
x-microsoft-antispam-prvs: <BY1PR03MB13399F029ADDB3FEC0D98BAC87EA0@BY1PR03MB1339.namprd03.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(158342451672863)(79290750141951);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(61425038)(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026)(61426038)(61427038); SRVR:BY1PR03MB1339; BCL:0; PCL:0; RULEID:; SRVR:BY1PR03MB1339;
x-forefront-prvs: 0044C17179
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(6009001)(7916002)(199003)(189002)(377454003)(13464003)(101416001)(8676002)(586003)(2906002)(74316002)(6116002)(2950100001)(102836003)(106116001)(99286002)(105586002)(2900100001)(7696003)(7736002)(7846002)(305945005)(92566002)(77096005)(122556002)(86612001)(76576001)(5002640100001)(15975445007)(19580395003)(106356001)(9686002)(86362001)(54356999)(50986999)(8936002)(76176999)(81166006)(81156014)(33656002)(10090500001)(5001770100001)(3280700002)(8990500004)(87936001)(19580405001)(107886002)(97736004)(5005710100001)(10290500002)(68736007)(3660700001)(10400500002)(15974865002)(5660300001)(189998001)(3826002); DIR:OUT; SFP:1102; SCL:1; SRVR:BY1PR03MB1339; H:BY1PR03MB1338.namprd03.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en;
received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Aug 2016 18:40:52.7677 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR03MB1339
Received-SPF: pass client-ip=104.47.40.96; envelope-from=Michael.Bishop@microsoft.com; helo=NAM03-CO1-obe.outbound.protection.outlook.com
X-W3C-Hub-Spam-Status: No, score=-3.9
X-W3C-Hub-Spam-Report: AWL=-2.420, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_NW=0.5
X-W3C-Scan-Sig: lisa.w3.org 1bcd74-0004ta-0r aa218a4ed63e1a40a40854ed82fa991a
X-Original-To: ietf-http-wg@w3.org
Subject: RE: Server Push and Content Negotiation
Archived-At: <http://www.w3.org/mid/BY1PR03MB1338B77F08376669853F92D487EA0@BY1PR03MB1338.namprd03.prod.outlook.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/32359
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

(From memory, without consulting devs)

On the server-side, there's an additional complication you're eliding over here -- the push may be cross-app, even on the same origin.  Say you have an app that dynamically renders things, but pushes other resources that are handled by a static file handler elsewhere in the system's request routing structure.  We essentially punt this choice to the app -- the app sends us a set of request headers.  We then simultaneously dispatch the PUSH_PROMISE on the wire and begin routing that request as if it had just been received from the client.  So whatever negotiation headers are specified by the app are those which will be seen by the recipient of the pushed request.  Requiring that we inspect the response to the push *before constructing* the request would require a total rearchitecting -- essentially requiring that the app hand us a fully-formed response and only then being able to declare the push.  That's going to delay processing of the initial request, which we really don't want blocked.

On the client side, there are two stages -- first, we compare the pushed request with the cache to see if we would have been able to serve that request out of cache.  If so, we reset the stream.  Otherwise, we receive and hold the content.  When the browser makes a request, we perform the same checks of whether the browser's request could be served by the held response.  So yes -- Vary is definitely necessary.

-----Original Message-----
From: Mark Nottingham [mailto:mnot@mnot.net] 
Sent: Tuesday, August 23, 2016 9:27 PM
To: HTTP Working Group <ietf-http-wg@w3.org>
Subject: Server Push and Content Negotiation

The interaction of Content Negotiation and Server Push isn't really specified. Depending on how it's implemented, it could be quite tricky, because it seemingly requires the server to guess what the client would have sent, in order to negotiate upon it.

However, it becomes much simpler if we assume that the client SHOULD NOT check a `PUSH_PROMISE` request's headers to see whether or not it would have sent that request.

This means, for example, that if you `PUSH_PROMISE` the "wrong" `User-Agent`, `Accept-Encoding`, `User-Agent` or even `Cookie` header field, the client SHOULD still use the pushed response; all they're looking for is a matching request method and URL.

However, this does imply a few things:

* The pushed request and response MUST still "agree"; i.e., if you're using gzip encoding, `Accept-Encoding` and `Content-Encoding` should be pushed with appropriate values.
* The pushed response MUST have an appropriate `Vary` header field, if it is cacheable. This is so that the cache operates properly.

Additionally, the server needs to know what the base capabilities and preferences of the client are, to allow it to select the appropriate responses to push. To aid this, servers SHOULD create a PUSH_PROMISE's request by copying the values of the request header fields mentioned in the `Vary` response header field from the request that is identified by the `PUSH_PROMISE` frame's Stream ID.

So, for example, if the first request for a page had the following headers:

~~~
:method: GET
:scheme: https
:authority: www.example.com
:path: /
User-Agent: FooAgent/1.0
Accept-Encoding: gzip, br
Accept-Language: en, fr
Accept: text/html,s application/example, image/*
Cookie: abc=123
~~~

and the server wishes to push these response headers for `/style.css`:

~~~
:status: 200
Content-Type: text/css
Cache-Control: max-age=3600
Content-Encoding: gzip
Vary: Accept-Encoding
~~~

then it should use these headers for the `PUSH_PROMISE`:

~~~
:method: GET
:scheme: https
:authority: www.example.com
:path: /style.css
Accept-Encoding: gzip, br
~~~

This approach has its limits. For example, use of Client Hints might not be practical with server push (since in some circumstances, hints might change between the base page request and the request for what's been pushed).

I'd be tempted to go even further and say that PUSH_PROMISE headers SHOULD NOT contain `DNT`, `User-Agent`, `Cookie` or similar headers UNLESS they were specified in Vary.

What do people think -- is this worth specifying? How are implementations currently doing this? 


--
Mark Nottingham   https://www.mnot.net/