Re: [jose] #27: member names MUST be unique needs additional text

Mike Jones <Michael.Jones@microsoft.com> Thu, 27 June 2013 23:19 UTC

Return-Path: <Michael.Jones@microsoft.com>
X-Original-To: jose@ietfa.amsl.com
Delivered-To: jose@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8E3321F9BAF for <jose@ietfa.amsl.com>; Thu, 27 Jun 2013 16:19:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.555
X-Spam-Level:
X-Spam-Status: No, score=-3.555 tagged_above=-999 required=5 tests=[AWL=0.043, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WmDWdyWJI5ZJ for <jose@ietfa.amsl.com>; Thu, 27 Jun 2013 16:19:18 -0700 (PDT)
Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0237.outbound.protection.outlook.com [207.46.163.237]) by ietfa.amsl.com (Postfix) with ESMTP id C911621F9982 for <jose@ietf.org>; Thu, 27 Jun 2013 16:19:17 -0700 (PDT)
Received: from BN1AFFO11FD018.protection.gbl (10.58.52.201) by BN1AFFO11HUB049.protection.gbl (10.58.52.160) with Microsoft SMTP Server (TLS) id 15.0.717.3; Thu, 27 Jun 2013 23:19:14 +0000
Received: from TK5EX14HUBC101.redmond.corp.microsoft.com (131.107.125.37) by BN1AFFO11FD018.mail.protection.outlook.com (10.58.52.78) with Microsoft SMTP Server (TLS) id 15.0.707.0 via Frontend Transport; Thu, 27 Jun 2013 23:19:14 +0000
Received: from TK5EX14MBXC283.redmond.corp.microsoft.com ([169.254.2.25]) by TK5EX14HUBC101.redmond.corp.microsoft.com ([157.54.7.153]) with mapi id 14.03.0136.001; Thu, 27 Jun 2013 23:18:35 +0000
From: Mike Jones <Michael.Jones@microsoft.com>
To: Richard Barnes <rlb@ipv.sx>
Thread-Topic: [jose] #27: member names MUST be unique needs additional text
Thread-Index: AQHOcW13mt9O8ZWU/EKeLtDRf+ylY5lHKNsAgAEqlQCAAA6YkIAAhRqAgAAMGPCAABbogIAABEiAgAAKPgCAAAP6AIAAlCSAgAAIEICAAAw40IAAaLoAgAABBaCAAAFPAIAAACRAgAAD8gCAAAH24A==
Date: Thu, 27 Jun 2013 23:18:34 +0000
Message-ID: <4E1F6AAD24975D4BA5B1680429673943678A1F90@TK5EX14MBXC283.redmond.corp.microsoft.com>
References: <061.bb7bbe0b618ec6b74904f48bdb9bb312@trac.tools.ietf.org> <076.a597050ecb4fb25084cec65f7174dc7e@trac.tools.ietf.org> <033b01ce729b$26ff5c90$74fe15b0$@augustcellars.com> <4E1F6AAD24975D4BA5B16804296739436789B4CC@TK5EX14MBXC283.redmond.corp.microsoft.com> <03b401ce72e5$002c65f0$008531d0$@augustcellars.com> <4E1F6AAD24975D4BA5B16804296739436789D4EA@TK5EX14MBXC283.redmond.corp.microsoft.com> <CAHBU6ivNow2xhDvRj7gG2gmHQ+C-ejEHCQubK=LwUtrXxdW6MA@mail.gmail.com> <255B9BB34FB7D647A506DC292726F6E1151BE45737@WSMSG3153V.srv.dir.telstra.com> <CAHBU6itHZ4NvGLyRqfDFaC9gkWD1_6dxhY=uVmwpyL+5Ay+j2w@mail.gmail.com> <255B9BB34FB7D647A506DC292726F6E1151BEDDD3C@WSMSG3153V.srv.dir.telstra.com> <CAHBU6iss5JbmHAue_2-AFDYSMVjAEnMLeLcESei3xTGTWP8_jQ@mail.gmail.com> <16F5858A-B77F-498B-9296-A42F84D5244D@ve7jtb.com> <4E1F6AAD24975D4BA5B1680429673943678A009C@TK5EX14MBXC283.redmond.corp.microsoft.com> <CAL02cgT4y4_zMC67ZKD5b=8JQHLVzuZSDP9ThiBF2Y_HYzMi5A@mail.gmail.com> <4E1F6AAD24975D4BA5B1680429673943678A1B91@TK5EX14MBXC283.redmond.corp.microsoft.com> <CAL02cgS3Hq2EGH9=+UH9wssfnsB9=Ns43jbTjhzqX9cMrhwtvw@mail.gmail.com> <4E1F6AAD24975D4BA5B1680429673943678A1CF7@TK5EX14MBXC283.redmond.corp.microsoft.com> <CAL02cgTYqyUpARxYZYupNMxQvE_WH4JgsQXGX1G_kkHdNxUpTQ@mail.gmail.com>
In-Reply-To: <CAL02cgTYqyUpARxYZYupNMxQvE_WH4JgsQXGX1G_kkHdNxUpTQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [157.54.51.72]
Content-Type: multipart/alternative; boundary="_000_4E1F6AAD24975D4BA5B1680429673943678A1F90TK5EX14MBXC283r_"
MIME-Version: 1.0
X-Forefront-Antispam-Report: CIP:131.107.125.37; CTRY:US; IPV:CAL; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(199002)(189002)(377454003)(24454002)(43544003)(377424004)(16236675002)(65816001)(50986001)(47976001)(6806003)(4396001)(49866001)(33656001)(512954002)(80022001)(47736001)(66066001)(63696002)(20776003)(16406001)(81542001)(31966008)(77096001)(71186001)(81342001)(69226001)(74502001)(74706001)(47446002)(54316002)(74662001)(19300405004)(15202345003)(56776001)(51856001)(76482001)(76786001)(53806001)(54356001)(79102001)(74366001)(76796001)(59766001)(77982001)(56816003)(46102001)(55846006)(74876001); DIR:OUT; SFP:; SCL:1; SRVR:BN1AFFO11HUB049; H:TK5EX14HUBC101.redmond.corp.microsoft.com; CLIP:131.107.125.37; RD:InfoDomainNonexistent; A:1; MX:1; LANG:en;
X-OriginatorOrg: microsoft.onmicrosoft.com
X-O365ENT-EOP-Header: Message processed by - O365_ENT: Allow from ranges (Engineering ONLY)
X-Forefront-PRVS: 08902E536D
Cc: Jim Schaad <ietf@augustcellars.com>, Tim Bray <tbray@textuality.com>, "jose@ietf.org" <jose@ietf.org>, John Bradley <ve7jtb@ve7jtb.com>, "Manger, James H" <James.H.Manger@team.telstra.com>, "draft-ietf-jose-json-web-signature@tools.ietf.org" <draft-ietf-jose-json-web-signature@tools.ietf.org>
Subject: Re: [jose] #27: member names MUST be unique needs additional text
X-BeenThere: jose@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Javascript Object Signing and Encryption <jose.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jose>, <mailto:jose-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/jose>
List-Post: <mailto:jose@ietf.org>
List-Help: <mailto:jose-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jose>, <mailto:jose-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jun 2013 23:19:25 -0000

It's not lossless only for illegal inputs - those with duplicate members.  If your goal isn't to defend against attackers, then it's safe to assume that the producer produced a legal input - hence no problem in practice.

From: Richard Barnes [mailto:rlb@ipv.sx]
Sent: Thursday, June 27, 2013 4:11 PM
To: Mike Jones
Cc: John Bradley; Tim Bray; Jim Schaad; Manger, James H; jose@ietf.org; draft-ietf-jose-json-web-signature@tools.ietf.org
Subject: Re: [jose] #27: member names MUST be unique needs additional text

The goal here is not to defend against attackers.  It's to prevent benign intermediaries from causing breakage.

The point here is that a JOSE implementation needs to get the same understanding of a JOSE object regardless of which of the multiple equivalent forms of the JSON form it gets.  This is about having a well-specified protocol, for which transformations that produce equivalent JSON objects should produce equivalent JOSE objects.  That is, the following process should be lossless:
JOSE --stringify--> JSON --reorder-fields--> JSON --parse--> JOSE

If that's not the case, then we've built a broken protocol.

--Richard







On Thu, Jun 27, 2013 at 7:00 PM, Mike Jones <Michael.Jones@microsoft.com<mailto:Michael.Jones@microsoft.com>> wrote:
Yes - but this is only a problem for the JSON Serialization - and then, only for fields that are not integrity protected.

If you don't care enough about the content of the header fields to integrity protect them, you clearly don't care whether an intermediary can tamper with them.  Reordering duplicate member names is the only the tip of the iceberg of the tampering that not integrity protecting header fields enables.  The intermediary could delete them or completely rewrite them!

Defending only against field reordering is like putting your finger in a hole in the dike when the floodgates next door are already wide open.  Your field will definitely flood when the sea rises.

                                                                -- Mike

From: Richard Barnes [mailto:rlb@ipv.sx<mailto:rlb@ipv.sx>]
Sent: Thursday, June 27, 2013 3:56 PM

To: Mike Jones
Cc: John Bradley; Tim Bray; Jim Schaad; Manger, James H; jose@ietf.org<mailto:jose@ietf.org>; draft-ietf-jose-json-web-signature@tools.ietf.org<mailto:draft-ietf-jose-json-web-signature@tools.ietf.org>
Subject: Re: [jose] #27: member names MUST be unique needs additional text

In a JSON-formatted JWS, the top-level JSON fields are not covered by the signature (only the protected header).  See the examples in my previous mail and in John's.
--Richard

On Thu, Jun 27, 2013 at 6:51 PM, Mike Jones <Michael.Jones@microsoft.com<mailto:Michael.Jones@microsoft.com>> wrote:
If an intermediary reorders the fields the signature validation will fail, so I'm not worried about that.

                                                                -- Mike

From: Richard Barnes [mailto:rlb@ipv.sx<mailto:rlb@ipv.sx>]
Sent: Thursday, June 27, 2013 3:48 PM
To: Mike Jones
Cc: John Bradley; Tim Bray; Jim Schaad; Manger, James H; jose@ietf.org<mailto:jose@ietf.org>; draft-ietf-jose-json-web-signature@tools.ietf.org<mailto:draft-ietf-jose-json-web-signature@tools.ietf.org>

Subject: Re: [jose] #27: member names MUST be unique needs additional text

On Thu, Jun 27, 2013 at 12:42 PM, Mike Jones <Michael.Jones@microsoft.com<mailto:Michael.Jones@microsoft.com>> wrote:
I've come to believe that James' proposed language does a good job of making security-appropriate restrictions while allowing the use of existing parsers.  It was:

A creator of a JOSE message MUST NOT put duplicate names in any JSON object in a JOSE header.  To prevent attacks where a JOSE message is interpreted as different valid messages by different recipients, each recipient MUST either reject messages with duplicate names or accept only the last name.

I might change "or accept only the last name" to "or use a parser that always uses the lexically last name for any duplicate member names, per [ECMAScript]" for clarity, but the intent is the same.

Would that work for people?

                                                            -- Mike


That's close.  I don't think we can allow the "lexically last" because elements in a JSON dictionary can be re-ordered.  Suppose there a helpful intermediary that sorts a JSON dictionary in descending alphabetical order of values.
When it gets: { "payload": "bad stuff",  "payload": "good stuff", ... }
It emits: { "payload": "good stuff",  "payload": "bad stuff", ... }

The only way to avoid the ambiguity is to reject objects with duplicate keys.


I realize that this renders most JSON parsers incompatible.  Implementors will need to add a secondary check.  I feel your pain.  But hopefully this pain will be temporary; the ambiguity will be fixed by the JSON WG and ECMA, and parsers will be updated.  Even in the short run, it doesn't seem like it would be that hard to mod existing parsers to have a "strict" flag.

"""
A creator of a JOSE message MUST NOT put duplicate names in any JSON object in a JOSE header.  To prevent attacks where a JOSE message is interpreted as different valid messages by different recipients, each recipient MUST reject messages.  Implementations MUST NOT select the lexically last name, as required by [ECMAScript], since this could lead them to select the incorrect value if an intermediary re-orders the fields in a dictionary.  If an implementation uses a JSON parser that does not reject objects with duplicate fields, then it MUST verify that there are no dictionaries with duplicate fields in the JSON object.
"""

--Richard






From: John Bradley [mailto:ve7jtb@ve7jtb.com<mailto:ve7jtb@ve7jtb.com>]
Sent: Thursday, June 27, 2013 8:49 AM
To: Tim Bray
Cc: Manger, James H; Mike Jones; Jim Schaad; jose@ietf.org<mailto:jose@ietf.org>; draft-ietf-jose-json-web-signature@tools.ietf.org<mailto:draft-ietf-jose-json-web-signature@tools.ietf.org>

Subject: Re: [jose] #27: member names MUST be unique needs additional text

The problem is not wanting to leave a hole for malicious signers to create JWT that may be misinterpreted by the receiver.  Accidental interop problems from bad signers are not a big concern for me and are covered by MUST NOT emit dups.

Accepting malformed JSON as the envelope may lead to real security issues.

This may be more of an issue for the JSON encoding than the compact as there you need to be careful the body you process for the signature is the body that the application uses.

What happens if you receive.


{"protected":<integrity-protected shared header contents>",

      "unprotected":<non-integrity-protected shared header contents>",

      "payload":"<original payload contents>",

      "payload":"<bad payload contents inserted by attacker>"

      "signatures":[

       {"header":"<per-signature unprotected header 1 contents>",

        "signature":"<signature 1 contents>"},

       ...

       {"header":"<per-signature unprotected header N contents>",

        "signature":"<signature N contents>"}],

     }

If the JOSE library validates on the first version of "payload" and the application using a different JSON parser only sees the second "payload" we have a wrapping attack similar to xmldsig.

I can't off the top of my head think of what you could do in the compact serialization that would lead to a successful attack given that the signature is over the base64url encoded segments, that prevents attackers from modifying the JSON while maintaining integrity.

It is a real issue for the JSON serialization though.

John B.

On 2013-06-27, at 11:20 AM, Tim Bray <tbray@textuality.com<mailto:tbray@textuality.com>> wrote:

The approach just seems wrong.  If we require that conforming implementations MUST NOT emit dupes, then authors of receiving software can just pick the best-performing parser with the nicest API, because they *all* perform correctly when there are no dupes.

Who's going to own the responsibility for making the authoritative finding that of the many JSON parsers in the Java ecosystem, these ones, but not those ones, are usable in JOSE applications?  And suppose one that has been previously OK is quietly optimized on GitHub to run 30% faster and as a side-effect silently ignores dupes?
The benefit of using JSON is that there is widely deployed software to handle it.  It is a known problem that sloppy spec drafting has allowed various kinds of problems to occur when dupe keys are received. "Doctor, it hurts when I do this." "So... don't do that."
Are we also going to specify the behavior of receiving software when the JSON has a misplaced quotation mark?  A missing trailing "}"?
 -T


On Wed, Jun 26, 2013 at 11:29 PM, Manger, James H <James.H.Manger@team.telstra.com<mailto:James.H.Manger@team.telstra.com>> wrote:
Most JSON libraries that silently accept [sob] dupe keys at least consistently use the last key. Allowing just that behaviour (or rejecting dupes), while forbidding acceptance of a non-last dupe (eg first key), is safe and allows most JSON libraries to be used. Since this accepts most libraries I think it is practical.

Hence I suggest:


>   A creator of a JOSE message MUST NOT put duplicate names in any JSON

> object in a JOSE header.

>   To prevent attacks where a JOSE message is interpreted as different

> valid messages by different recipients, each recipient MUST either

> reject messages with duplicate names or accept only the last name.

This isn't "truly predictable" as a message with dupes might be accepted or rejected. The crucial point is that if a message is accepted it is predictable.

--
James Manger

From: Tim Bray [mailto:tbray@textuality.com<mailto:tbray@textuality.com>]
Sent: Thursday, 27 June 2013 4:16 PM
To: Manger, James H
Cc: Mike Jones; Jim Schaad; jose@ietf.org<mailto:jose@ietf.org>; draft-ietf-jose-json-web-signature@tools.ietf.org<mailto:draft-ietf-jose-json-web-signature@tools.ietf.org>

Subject: Re: [jose] #27: member names MUST be unique needs additional text


Well, you have a practical problem in that most implementers will want to use a standard JSON library, which is good practice because it will be well-debugged, and most libraries [sob] silently take care of dupe keys and don't have a way to tell the client software what happened. So if you want truly predictable behavior, you're forcing the use of hand-constructed JSON parsers. And that sucks, because getting good performance in JSON parsing is surprisingly hard, with dramatic performance differences between implementations. So you're forcing receiving software which wants to be conformant to use a hand-rolled parser which will probably have lousy performance and have other bugs which in fact may compromise security more than dupe-key tricks could.  -T

On Wed, Jun 26, 2013 at 10:39 PM, Manger, James H <James.H.Manger@team.telstra.com<mailto:James.H.Manger@team.telstra.com>> wrote:
> I think it's a nice clean minimal solution to say that producers MUST
> NOT generate dupes, end of story.  I don't think saying anything beyond
> that adds value. -T
Clean and minimal that may be, but it ignores the security issue. We don't want a malicious producer (who is so malicious they ignore a MUST) to create JOSE messages that a JOSE-compliant security layer accepts as "benign interpretation #1" so it passes the message on to the JOSE-compliant backend app that acts on "nasty interpretation #2".

--
James Manger




_______________________________________________
jose mailing list
jose@ietf.org<mailto:jose@ietf.org>
https://www.ietf.org/mailman/listinfo/jose