Re: [OAUTH-WG] HTTP request signing and repeated query/header keys

Justin Richer <jricher@mit.edu> Tue, 01 March 2016 22:02 UTC

Return-Path: <jricher@mit.edu>
X-Original-To: oauth@ietfa.amsl.com
Delivered-To: oauth@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7F4EB1B423C for <oauth@ietfa.amsl.com>; Tue, 1 Mar 2016 14:02:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.206
X-Spam-Level:
X-Spam-Status: No, score=-4.206 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.006, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HLvmN0vpgIQu for <oauth@ietfa.amsl.com>; Tue, 1 Mar 2016 14:02:36 -0800 (PST)
Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu [18.9.25.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3DD8C1B4257 for <OAuth@ietf.org>; Tue, 1 Mar 2016 14:02:36 -0800 (PST)
X-AuditID: 1209190c-59fff70000006700-e3-56d6117a8530
Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id A7.FB.26368.A7116D65; Tue, 1 Mar 2016 17:02:34 -0500 (EST)
Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id u21M2X2W030230; Tue, 1 Mar 2016 17:02:34 -0500
Received: from [10.0.1.8] (ip68-104-118-32.lv.lv.cox.net [68.104.118.32]) (authenticated bits=0) (User authenticated as jricher@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id u21M2R8n005980 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 1 Mar 2016 17:02:30 -0500
Content-Type: multipart/alternative; boundary="Apple-Mail=_46F46E45-1F11-4E7C-9956-091B07F2FE4B"
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
From: Justin Richer <jricher@mit.edu>
In-Reply-To: <C91096A6-227E-49EA-9813-95A2F0704CA6@oracle.com>
Date: Tue, 1 Mar 2016 14:02:26 -0800
Message-Id: <693D8AD3-E9E7-4DBB-8466-215C2030BEE5@mit.edu>
References: <003b01d1726f$66036590$320a30b0$@gmail.com> <16802C28-5478-41D5-8596-30C617C89016@mit.edu> <005401d17272$070325a0$150970e0$@gmail.com> <844C70FF-8D41-4D4D-8F9D-F1CAC5D69D9E@mit.edu> <006f01d1728e$f52af250$df80d6f0$@gmail.com> <255B9BB34FB7D647A506DC292726F6E13BBCD3D02F@WSMSG3153V.srv.dir.telstra.com> <C91096A6-227E-49EA-9813-95A2F0704CA6@oracle.com>
To: Phil Hunt <phil.hunt@oracle.com>
X-Mailer: Apple Mail (2.2104)
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLKsWRmVeSWpSXmKPExsUixCmqrVsleC3M4Nw3bYsZP46yW2y984rR 4uTbV2wWC+Y3sjuweOycdZfdY8mSn0weH5/eYvGY17GBOYAlissmJTUnsyy1SN8ugSvj6t61 bAX9XxgrXvdcZmtgfHiNsYuRk0NCwETi4tUDQDYXh5BAG5PE3turmCCcDYwSJz++g8ocZZI4 P3kCO0gLs0CCxO1l88DaeQX0JF7duswKYgsLeEv8PdQPZrMJqEpMX9PCBGJzCthJHF7xFKye RUBFYtnl9ewgQ5kFehklTl1dATXISmLKv4UsENteM0lM2DIRbJIIUMe3q9ehjpWV2P37EdME Rv5ZSA6ZheQQiLi2xLKFr5khbE2J/d3LWTDFNSQ6v01kXcDItopRNiW3Sjc3MTOnODVZtzg5 MS8vtUjXUC83s0QvNaV0EyM4FiR5djCeeeN1iFGAg1GJhzfj09UwIdbEsuLK3EOMkhxMSqK8 fceBQnxJ+SmVGYnFGfFFpTmpxYcYJTiYlUR4P38DyvGmJFZWpRblw6SkOViUxHkL958OExJI TyxJzU5NLUgtgsnKcHAoSfCqCFwLExIsSk1PrUjLzClBSDNxcIIM5wEaHgtSw1tckJhbnJkO kT/FqCglzhsNkhAASWSU5sH1glKVS0aZwitGcaBXhHl3gFTxANMcXPcroMFMQIMl1l8CGVyS iJCSamBct+/guT8bvvns8mdcZS9S5hVnnbBX7m5uk5RWx8kak0vXeObOkvZ8wfqeaZWWh1Tv pcmOs4uKuzSPRPVKr/W2sEzayf+DoTE1Y2f+3e/hn5e+W7Y6KXbzs7YqW7/vpSbP7b8fElwZ 76IcvXjTi5lLV87L3DrjiY3lgpVaPTdDV6l97wh5ypqmxFKckWioxVxUnAgAVy5GVDADAAA=
Archived-At: <http://mailarchive.ietf.org/arch/msg/oauth/CqF1hM11c9l_sQw7GZDQz3w_T0w>
Cc: "<oauth@ietf.org>" <OAuth@ietf.org>
Subject: Re: [OAUTH-WG] HTTP request signing and repeated query/header keys
X-BeenThere: oauth@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: OAUTH WG <oauth.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/oauth>, <mailto:oauth-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/oauth/>
List-Post: <mailto:oauth@ietf.org>
List-Help: <mailto:oauth-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/oauth>, <mailto:oauth-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 01 Mar 2016 22:02:40 -0000

+1, this was a driving requirement when I wrote the first strawman. I can’t tell you the number of times I had frameworks mess things up with OAuth 1, which does exactly the algorithm that you mention below. 

I’m currently in favor of just leaving the repeated parameter and header out of the core hashing mechanism. We could, if someone wanted to get really clever, introduce a more generic hashing mechanism that could handle the whole message in different ways. But I think the existing mechanism is good enough for most cases that we can roll with it.

 — Justin


> On Feb 28, 2016, at 7:18 PM, Phil Hunt (IDM) <phil.hunt@oracle.com> wrote:
> 
> The issue for any service is that things like gateways and proxies may change or add parameters.
> 
> There is a long tradition with http to mess with things headers and payload in the real world. Not ideal, but a certain reality. 
> 
> Thus for certain services it is likely only possible to sign a subset of params that the client and service provider want to verify did not change. 
> 
> Phil
> 
> On Feb 28, 2016, at 18:39, Manger, James <James.H.Manger@team.telstra.com <mailto:James.H.Manger@team.telstra.com>> wrote:
> 
>> Being able to selectively include or exclude individual query string name=value parameters in a signature feels far too dangerous. Always sign them all — with the only exception being the signature itself if transmitted as a query string parameter.
>> Unfortunately we do need to selectively include or exclude HTTP headers as they are a mix of transport, app-layer, end-to-end, and hop-by-hop items.
>>  
>> Ignoring duplicate query parameter names is a poor design for a fairly generic HTTP signing method. My suggestion would be to do a stable sort on parameter names. That is, sort the names, but preserve the order of values when a name appears multiple times. …hoping that works with almost all frameworks.
>>  
>> --
>> James Manger
>>  
>> From: OAuth [mailto:oauth-bounces@ietf.org <mailto:oauth-bounces@ietf.org>] On Behalf Of Brock Allen
>> Sent: Monday, 29 February 2016 12:17 PM
>> To: 'Justin Richer' <jricher@mit.edu <mailto:jricher@mit.edu>>
>> Cc: '<oauth@ietf.org <mailto:oauth@ietf.org>>' <OAuth@ietf.org <mailto:OAuth@ietf.org>>
>> Subject: Re: [OAUTH-WG] HTTP request signing and repeated query/header keys
>>  
>> Yea, I’m cool with just saying “duplicate keys won’t work” for my implementation, unless I do some implementation specific thing like sort them by value order or some such (but that would have to be on both sides).
>>  
>> -Brock
>>  
>> From: Justin Richer [mailto:jricher@mit.edu <mailto:jricher@mit.edu>] 
>> Sent: Sunday, February 28, 2016 8:13 PM
>> To: Brock Allen <brockallen@gmail.com <mailto:brockallen@gmail.com>>
>> Cc: <oauth@ietf.org <mailto:oauth@ietf.org>> <OAuth@ietf.org <mailto:OAuth@ietf.org>>
>> Subject: Re: [OAUTH-WG] HTTP request signing and repeated query/header keys
>>  
>> Yeah, that’s the trick — you don’t want to end up replicating the entire HTTP message inside the JWS envelope, because then you end up with a SOAP kind of approach where you’re not really using the outside HTTP constructs to their fullest anymore. 
>>  
>> In the end, I don’t think this spec is ever going to be perfectly universal, but if it can hit a majority of use cases with a simple and robust solution, I think we’re in good shape.
>>  
>>  — Justin
>>  
>> On Feb 28, 2016, at 1:50 PM, Brock Allen <brockallen@gmail.com <mailto:brockallen@gmail.com>> wrote:
>>  
>> Now that I’m thinking about this, the only thing that comes to mind is a hash of the value included somehow (which I’m sure you’ve already thought of). That’s obviously more complexity and overhead for all scenarios to support this edge case.
>>  
>> -Brock
>>  
>> From: Brock Allen [mailto:brockallen@gmail.com <mailto:brockallen@gmail.com>] 
>> Sent: Sunday, February 28, 2016 4:40 PM
>> To: 'Justin Richer' <jricher@mit.edu <mailto:jricher@mit.edu>>
>> Cc: '<oauth@ietf.org <mailto:oauth@ietf.org>>' <OAuth@ietf.org <mailto:OAuth@ietf.org>>
>> Subject: RE: [OAUTH-WG] HTTP request signing and repeated query/header keys
>>  
>> Ok, missed that – thanks.
>>  
>> For now in my implementation I’m also ignoring this problem :)
>>  
>> -Brock
>>  
>> From: Justin Richer [mailto:jricher@mit.edu <mailto:jricher@mit.edu>] 
>> Sent: Sunday, February 28, 2016 4:37 PM
>> To: Brock Allen <brockallen@gmail.com <mailto:brockallen@gmail.com>>
>> Cc: <oauth@ietf.org <mailto:oauth@ietf.org>> <OAuth@ietf.org <mailto:OAuth@ietf.org>>
>> Subject: Re: [OAUTH-WG] HTTP request signing and repeated query/header keys
>>  
>> In §7.5 we currently have:
>>  
>>    The behavior of repeated query parameters or repeated HTTP headers is
>>    undefined by this specification.  If a header or query parameter is
>>    repeated on either the outgoing request from the client or the
>>    incoming request to the protected resource, that query parameter or
>>    header name MUST NOT be covered by the hash and signature. [[ Note to
>>    the WG: is this something we need to cover? ]]
>>  
>> Which is to say: Yeah, that’s a problem, probably. We either declare this behavior out of scope and say you can’t use this method with that kind of API (or at least those parameters/headers) or we define a mechanism for handling that. I’m in favor of the former, and removing the text from the generation sections that says repeated parameters are processed with no special handling, making them explicitly out of scope throughout.
>>  
>> I would love to see more feedback from the group about this, especially if someone’s got a clever solution.
>>  
>>  — Justin
>>  
>> On Feb 28, 2016, at 1:31 PM, Brock Allen <brockallen@gmail.com <mailto:brockallen@gmail.com>> wrote:
>>  
>> Given that the client can iterate over the query/headers in any order to generate the concatenated value to hash, I think there’s an issue with query string or header values with repeated keys. I’ll stick with query params for simplicity in my sample.
>>  
>> If a client signs this concatenated query: “a=1&a=2” then the “p” value in the signed JSON object could be this:
>>  
>> "p": [["a", "a"], "blah"]
>>  
>> On the resource server, how do you know which key/value pair to put in which order for verification? Also, given that different server implementations express these incoming parameters in different ways, it seems problematic to be consistent.
>>  
>> Thanks.
>>  
>> -Brock
>>  
>> _______________________________________________
>> OAuth mailing list
>> OAuth@ietf.org <mailto:OAuth@ietf.org>
>> https://www.ietf.org/mailman/listinfo/oauth <https://www.ietf.org/mailman/listinfo/oauth>
>>  
>>  
>> _______________________________________________
>> OAuth mailing list
>> OAuth@ietf.org <mailto:OAuth@ietf.org>
>> https://www.ietf.org/mailman/listinfo/oauth <https://www.ietf.org/mailman/listinfo/oauth>