Re: [jose] JWS Unencoded Payload Option and the % character

[John Bradley <[e-mail]>]

I think I get Jim's point. 

If the receiver doesn't know what the content type of the body is,  it might be possible that it could misinterpret the message by decoding  when it shouldn't. 

That would have a message with a valid signature with two possible application level outputs. 

On the other hand how is that really different from someone sending % escaped characters inside the base64?  

People may have all sorts of additional encoding in the body that could result in a different message after additional decoding or not.  

If the signature is over the body without any escaping then if it is modified the signature will fail.   Interpreting how the body is processed at the application level is not unique to this extension. 

I don't think it is a new security concern. 

John B. 

Sent from my iPhone

> On Sep 29, 2015, at 2:20 AM, Mike Jones <[e-mail]> wrote:
> 
> I don’t think that’s the case.  Short of cryptographic collisions occurring, every distinct JWS Payload string will have a distinct signature, both in the “b64”:true and “b64”:false cases.
>  
> It is true that if the application chooses to interpret % to mean form-urlencoding at the application level, then it is true that multiple application-level payloads with the same value but different encodings could have different corresponding JWS Payloads, and therefore different signatures.  For instance, the distinct JWS Payload values “A” and “%41” would have different signatures but would be decoded by some applications to the same application-level payload.
>  
> But that’s the opposite of what you’re describing, Jim.  This is a case of the same application-level payload having multiple different encodings, and therefore multiple possible JWS Payloads, and therefore multiple possible corresponding signatures – not multiple payloads with the same signature.
>  
>                                                             -- Mike
>  
> From: Jim Schaad [mailto:[e-mail]] 
> Sent: Monday, September 28, 2015 9:53 PM
> To: 'Nat Sakimura'; 'John Bradley'
> Cc: Mike Jones; [e-mail]; 'Vladimir Dzhuvinov'
> Subject: RE: [jose] JWS Unencoded Payload Option and the % character
>  
>  
>  
> From: jose [mailto:[e-mail]] On Behalf Of Nat Sakimura
> Sent: Monday, September 28, 2015 5:40 PM
> To: John Bradley <[e-mail]>
> Cc: Michael Jones <[e-mail]>; [e-mail]; Vladimir Dzhuvinov <[e-mail]>
> Subject: Re: [jose] JWS Unencoded Payload Option and the % character
>  
> Being able to express UTF-8 characters in %encoding is not particularly appealing since you cannot read the % encoded strings and it will be longer than base64url encoded strings, e.g. my name in 
>  
> %encoding: %E5%B4%8E%E6%9D%91%E5%A4%8F%E5%BD%A6
> b64url encoding: 5bSO5p2R5aSP5b2m
>  
> So, I suppose the use case for % encoding is in the case of predominantly US-ASCII characters payload with relatively small number of non-url safe characters. Like James, I feel a bit that it is a corner case, but since the semantics of "b64":false is that JW* layer takes the payload as-is and does not process it, why not let % be allowed and let the applications that use JW* decide what to do with it. 
>  
> OR -- would it cause some security concerns? 
>  
> [JLS] Minor security concern. If the application is not specified in the protected attributes, you now have two different strings which generate the same signature value depending on how you handle the % encoding.  I would prefer to say, don’t do this if you want to send via a URL.
>  
> Jim
>  
>  
> Nat
>  
> 2015-09-29 9:12 GMT+09:00 John Bradley <[e-mail]>:
> Thinking about this a bit, I don’t think we should impose any special semantics on % in JW* processing.
>  
>  
> It should be a part of the body that is opaque other than the constraints around character set.   If the result is sent via URL then it would be encoded and decoded at the applications risk when using b64 : false.
>  
> I think that is the simplest thing.   If they need to worry about encoding it then they should not be using b64: false.
>  
> If the application wants to receive a bunch of unicode characters that way then they can.  (They would have been better off b64 encoding them than URL but, different strokes.
>  
> John B.
>  
> On Sep 28, 2015, at 3:56 PM, Mike Jones <[e-mail]> wrote:
>  
> Thanks for your thoughtful comments, Vladimir.  The objective is to enable using unencoded payloads.
>  
> Thinking about it in terms of that goal, that actually eliminates choices 3, 4, and 5, because all require supporting a new payload encoding (x-www-form-urlencoded encoding), which defeats the purpose.
>  
> That leaves 1 (prohibit %) and 2 (allow % with no JWS-level processing performed).  Because 2 gives applications the flexibility to  use % for application-level encoding if they choose, I’m now thinking that 2 is probably the more general choice than 1.  The only caveat is that applications would have to be aware that when passed in URLs, % would have to be represented as %25, since it is not URL-safe.
>  
> What do people think of the choice between 1 and 2?
>  
>                                                                 -- Mike
>  
> From: jose [mailto:[e-mail]] On Behalf Of Vladimir Dzhuvinov
> Sent: Friday, September 25, 2015 8:29 AM
> To: [e-mail]
> Subject: Re: [jose] JWS Unencoded Payload Option and the % character
>  
> I read the entire discussion. I'm unsure how to rate the five choices. If I knew what the underlying objective is, it would be easier to make a technical judgment. 
> 
> So what are we trying to achieve here?
> 
> Allow web apps to pass around such JWS messages more easily? Then URL-safety would matter. But how likely is this use case?
> 
> Or save apps additional processing?
> 
> Or keep the JWS payload as unmodified as possible?
> 
> Could this be left to the actual app to determine, and hence the most suitable encoding?
> 
> Vladimir
> 
> On 23.09.2015 05:41, Manger, James wrote:
> Comments inline
>  
> From: Mike Jones [mailto:[e-mail]]
> Sent: Wednesday, 23 September 2015 11:52 AM
> To: Manger, James; [e-mail]
> Subject: RE: JWS Unencoded Payload Option and the % character
>  
> Writing as an individual, your possible option 5 has the following downsides, at least as I see it:
>  
> (a) It doesn't represent the payload in unencoded form, which was one of the primary motivations for this option.
>  
> The payload can be unencoded in the JSON Serialization and when detached - those are the motivations; not an unencoded form in a non-detached Compact Serialization.
>  
> (b) It's not self-consistent, since the "b64":false treatment of detached payloads requires them to be unencoded whereas the treatment of attached payloads requires the opposite.
>  
> It is consistent in always treating the 2nd part of a JWS as a base64url-encoded payload. The consistency of handling detached payloads depends on the API you offer. If your API expects "detachedSigningInput" it will depend on "b64" (ie be inconsistent). However, if your API expects "detachedRawPayload" it does not depend on "b64" (ie will be consistent). The latter makes more sense to me: RawPayload is something the call cares about; SigningInput is an internal detail.
>  
> (c) It breaks the invariant that the JWS Signing Input is simply the contents of the JWS prior to the second period - which is one of the simple things about JWSs using the compact serialization.
>  
> That is not a particularly useful invariant. It is not as though a signature verification sub-system can treat a "JWS prior to the 2nd period" as a blob. It still needs to split on the period, base64url-decode the 1st part, JSON-parse it, then look at the algorithm & key id, before doing any verification.
>  
> And I don't see any particular upside.  It's just a standard JWS with an unnecessarily different JWS Signing Input computation but the same payload representation and an extra field in the encoded header representation.  Better to just use a normal JWS, given the choice between that and 5.
>  
> Indeed, "b64":false is unnecessary if you are using non-detached Compact Serializations. The upside is for large payloads that are detached or use the JSON Serialization.
>  
> --
> James Manger
>  
>  
> From: Manger, James [mailto:[e-mail]]
> Sent: Tuesday, September 22, 2015 6:05 PM
> To: Mike Jones; [e-mail]<mailto:[e-mail]>
> Subject: RE: JWS Unencoded Payload Option and the % character
>  
> Or a 5th option:
>  
> 5. "b64":false affects the Signing Input, but not the Compact Serialization (which remains a URL-safe string for any Payload). The 2nd dot-separated component of the Compact Serialization is always BASE64URL(JWS Payload); a '%' in the Payload causes no issues, neither does a '.' nor any other octet.
>  
> The only corner case option 5 prevents is when you have: (1) a large payload; (2) that doesn't contain octet 0x2E '.'; (3) probably doesn't contain any of the other 190 octet values not in the URL-safe set; (4) you want to use the Compact Serialization; (5) you don't want to use a detached payload; and (6) you cannot tolerate the additional 33% space overhead from base64url-encoding the Payload. I don't think this is a corner case anyone is interested in.
>  
> --
> James Manger
>  
> From: jose [mailto:[e-mail]] On Behalf Of Mike Jones
> Sent: Wednesday, 23 September 2015 8:23 AM
> To: [e-mail]<mailto:[e-mail]>
> Subject: [jose] JWS Unencoded Payload Option and the % character
>  
> There's one outstanding issue with the JWS Unencoded Payload Option specification that I'd like to see working group discussion on:  What should the processing rules be for a '%' character in the JWS Payload for a non-detached payload using "b64":false with the JWS Compact Serialization?  I see the possibilities as being:
>  
> 1.  Use of '%' is prohibited, because it is not URL-safe.  This is the behavior current specified in https://tools.ietf.org/html/draft-ietf-jose-jws-signing-input-options-02#section-5.2<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ftools.ietf.org%2fhtml%2fdraft-ietf-jose-jws-signing-input-options-02%23section-5.2&data=01%7c01%7cMichael.Jones%40microsoft.com%7c6fbe6ca0c59048e2d97808d2c3b2f875%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=tKBYn8mzSjJkpYNPR5JhiSqDrg8n9
>  M
> VFrV28pv%2fQGW8%3d>.  This is the simplest option.  It means that inline unencoded payloads are limited to using letters, numbers, dash, underscore, and tilde.
>  
> 2.  Use of '%' is allowed and has no defined semantics at the JWS level; it's just another allowed character.  This maintains the invariant that the JWS Signing input consists of the characters before the second '.' in the JWS representation.  Note that because '%' is not URL-safe, any URLs containing JWS containing '%' characters would have to form-url-encode them - resulting in them being represented in the URL as "%25".  Applications *could* use '%' at the application level to escape octets using the '%' <hex> <hex> convention but this escaping would not be understood by JWS.  For example, the JWS Payload could be "%24%2E02", be represented in the JWS as "%24%2E02", be represented in URLs as "%2524%252E02", and the JWS Signing Input would contain "%24%2E02".  I believe that this is the position that was being advocated by Sergey Beryozkin in http://www.ietf.org/mail-
>  a
> rchive/web/jose/current/msg05257.html<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.ietf.org%2fmail-archive%2fweb%2fjose%2fcurrent%2fmsg05257.html&data=01%7c01%7cMichael.Jones%40microsoft.com%7c6fbe6ca0c59048e2d97808d2c3b2f875%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=2o5YW1FQaTiawjuFSlY%2fizoWdF7jjTCq3QOgTW%2fuQ4Y%3d>.
>  
> 3.  Use of '%' is allowed and is used for '%' <hex> <hex> encoding of payload octets, with the JWS Signing Input keeping the '%' <hex> <hex> characters as-is.  This maintains the invariant that the JWS Signing input consists of the characters before the second '.' in the JWS representation.  It requires form-url-decoding of any payload value containing '%' when returning the JWS Payload.    For example, the JWS Payload could be "$.02", be represented in the JWS as "%24%2E02", be represented in URLs as "%2524%252E02", and the JWS Signing Input would contain "%24%2E02".
>  
> 4.  Use of '%' is allowed and is used for '%' <hex> <hex> encoding of payload octets, with the JWS Signing Input containing the encoded octets.  This loses the invariant that the JWS Signing input consists of the characters before the second '.' in the JWS representation.  It requires form-url-decoding of any payload value containing '%' both when doing signing and when returning the JWS Payload.    For example, the JWS Payload could be "$.02", be represented in the JWS as "%24%2E02", be represented in URLs as "%2524%252E02", and the JWS Signing Input would contain "$.02".  This is the most consistent with the JWS JSON Serialization processing rules in https://tools.ietf.org/html/draft-ietf-jose-jws-signing-input-options-02#section-5.3<https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2ftools.ietf.org%2fhtml%2fdraft-ietf-jose-jws-signing-input-options-02%23section-5.3&data=01%7c01%7cMichael.Jones%40microsoft.com%7c6fbe6ca0c59048e2d97808d2c3b2f875%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=wb%2fq6RyH2Oy1Km8PCIJmcDyz5gsQqBISJMKDvIy%2bIJg%3d>, in which the JWS Payload and JWS Signing Input values are determined after performing any escape processing.  I believe that this is the position that was being advocated by Jim Schaad in http://www.ietf.org/mail-archive/web/jose/current/msg05259.html<https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fwww.ietf.org%2fmail-archive%2fweb%2fjose%2fcurrent%2fmsg05259.html&data=01%7c01%7cMichael.Jones%40microsoft.com%7c6fbe6ca0c59048e2d97808d2c3b2f875%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=NR%2fLeoyOPWOoJO9%2bZsgrutgVAGBxLYZttVWQ8CPdG14%3d>.
>  
> How would working group members like to see us use (or not use) '%'?
>  
>                                                                 -- Mike
>  
>  
>  
> 
> _______________________________________________
> jose mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/jose
>  
> 
> -- 
> Vladimir Dzhuvinov :: [e-mail]
> _______________________________________________
> jose mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/jose
>  
> 
> _______________________________________________
> jose mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/jose
> 
> 
> 
>  
> --
> Nat Sakimura (=nat)
> Chairman, OpenID Foundation
> http://nat.sakimura.org/
> @_nat_en