Re: [jose] Whether implementations must understand all JOSE header fields

[Mike Jones <[e-mail]>]

About your "first", I understand that we may personally disagree about what the best answer to that question is.  Richard, John, and I had proposed the following consensus call question to bring closure to that issue.


FIRST POLL:  Should all header fields be critical for implementations to understand?

YES - All header fields must continue to be understood by implementations or the input must be rejected.

NO - A means of listing that specific header fields may be safely ignored should be defined.

About your "second", I believe that the text in the document is aligned with my statements, but it could be expanded upon if the working group believes that it would be useful.  In particular, we could add something like this in an appropriate place:

The requirement that implementations must understand all header parameter values is a requirement on the system as a whole - not on any particular component or layer of the implementation.  Should part of the implementation be contained in a library, and if there are be header parameters that the library doesn't understand, the responsibility for fulfilling the "must understand" requirement for the remaining header parameters would then fall to the application using that library.

I believe that that clarification would narrow our disagreement to the consensus call question above, which once posed the working group, we'll achieve closure on.  Hopefully we can do that in the next week or so, once people are likely to be back from their holiday break.

                                                            Best wishes,
                                                            -- Mike

From: Jim Schaad [mailto:[e-mail]]
Sent: Monday, December 31, 2012 11:44 AM
To: Mike Jones; [e-mail]
Subject: RE: [jose] Whether implementations must understand all JOSE header fields

I would say probably not.

First, I would allow applications to state that it is not necessary for all header fields to be understood.

Second, I don't believe that the text in the document matches what you have said below.

Jim


From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]] On Behalf Of Mike Jones
Sent: Sunday, December 30, 2012 12:21 AM
To: Jim Schaad; [e-mail]<mailto:[e-mail]>
Subject: Re: [jose] Whether implementations must understand all JOSE header fields

"Must understand" is a requirement on the system as a whole - not on a particular level of library software.  It's perfectly fine for a JOSE library to process the headers that it understands and then leave the processing of the rest of them up to the application - which I think is what you are advocating.  For those headers that the JOSE library doesn't understand, the responsibility for fulfilling the "must understand" requirement for the remaining headers would then fall to the application.

I really don't think this is far from what you're thinking of.  I completely agree with you that inflexible libraries that reject the input because they don't understand all the header values in it without giving the application a chance to weigh are probably not good software design.

The spec intentionally doesn't say anything about implementation layering, nor probably should it.  It imposes requirements on the complete, end-to-end system - not particular implementation layers.  At least as I see it, that's an inclusive view of what we're trying to accomplish - not a narrow one.

Does that understanding move us closer to agreement?  I hope so...

                                                            Best wishes,
                                                            -- Mike

From: Jim Schaad [mailto:[e-mail]]
Sent: Saturday, December 29, 2012 9:59 PM
To: Mike Jones; [e-mail]<mailto:[e-mail]>
Subject: RE: [jose] Whether implementations must understand all JOSE header fields

I am going to go into more detail than my last message

The issue is a question of which part of the code is responsible for what set of processing.

If a library says that an item is either encrypted or signed and it wasn't, or it was unable to apply those transformations on request, then you have a security problem.  If you hand the library a piece of garbage and it says that it was not signed (or encrypted) then you do not have a security problem, you have a different type of problem.

The same thing is true with the ZIP example you gave earlier.  Is the application or the JW* library responsible for dealing with the question of zipping the content?  If you say the application is responsible, then it does not matter of the JW* code does not recognize the ZIP parameter.  If you say the JW* library is responsible, then it does matter if the JW* code does not recognize the ZIP parameter.  You then also have the question should the JW* code have the "right" to apply a ZIP operation even if it was not asked for by the application.  It is after all part of the library and thus could be applied at any time.

The position of mandatory understanding of fields is much more justifiable if one states that the library is responsible for enforcing everything about the header.  This would mean that the concept of a signing time field in the header would not make any sense.  The JW* library has no way of knowing what the semantics of the field is going to mean to the application and thus has  no method of enforcing it.  (Does the object expire a specific amount of time after the signing time?  Does there need to be some type of allowance for clock skews?  Is there an interaction between a certificate if given and the signing time as a header field?  Are there fields in the key container that say when a key can be used and thus need to be enforced?)

If on the other hand one believes that there is a possibility that there is auxiliary data which is not part of the content but is part of the context in which the signature (or encryption) was applied, it is impossible for a JW* library to be able to enforce any of this context and it must be deferred to the application for enforcement.

To directly address your points below:


1.       We owe developers a clear description of what to do with the items that we defined.  If an application defines another field, then it will defined what to do with that item and not the JW* code and the application would be expected to enforce the semantics and not the JW* code.  It does mean that each application will have its own list of what items can appear, however it does not mean that the JW* code cannot supply a library routine for doing the field name checking.

2.      I am completely unable to understand why allowing a field for one application and not for another application hurts interoperability or enforcement of the fields.   But as I am doing this in the application, I only have one standard that I am working with (until such a time as the application is updated).

3.      If a profile decides that the alg field need not be present, which is different that it's value not being understood, then that is up to the application.  As long as the application does not make a security claim about the content that as transmitted that is a perfectly fine thing for it to do.  It might just be transmitted as plain unsigned, unencrypted data in the same format as the current JW* formats.  There is a different issue involved with the value of the alg field not being understood.  That is something that the JW* libraries are perfectly competent to deal with and is not the same type of extension mechanism as we are discussing here.

I completely disagree that the option is either  not viable, or is a security problem.  I think that this is your narrow view of the world affecting how you are evaluating the proposal that I have provided.  However as long as you are going to stay in the camp that says there is no auxiliary data which is not part of the content of the message, then your view is a consistent one.  I happen to strongly disagree with that view of the world.

Jim




From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]] On Behalf Of Jim Schaad
Sent: Saturday, December 29, 2012 8:25 PM
To: 'Mike Jones'; [e-mail]<mailto:[e-mail]>
Subject: Re: [jose] Whether implementations must understand all JOSE header fields



From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]] On Behalf Of Mike Jones
Sent: Wednesday, December 26, 2012 2:32 PM
To: Jim Schaad; [e-mail]<mailto:[e-mail]>
Subject: Re: [jose] Whether implementations must understand all JOSE header fields

That idea seems like it would (1) add to the overall complexity of implementations, (2) would be counter the goal of interoperable implementations, and (3) would likely be counter to goal of security.

(1) The way I see it, if an extension mechanism (other than having both parties understand the extension, which already works) is going to be defined, we owe users of the specs a clear definition of how to do not-understood extensions.  Otherwise implementations of each profile will potentially have to repeat the work of checking the fields used, rather than doing it in the JOSE libraries themselves, adding to overall implementation complexity.

(2)  Sometimes allowing fields to be not understood and sometimes requiring them to be, and possibly using different extension mechanisms in different profiles would definitely hurt interoperability.

(3)  If a profile decided that the "alg" field need not be understood, that would definitely defeat the security purposes of the JOSE specs.  The same is true of many other fields.  We owe it to users of the specs to be clear that security-critical fields must be understood.  "Punting" this to profiles or just putting it in the security considerations unnecessarily risks having profile writers get this wrong.

[JLS]  I would be shocked if a JWS implementation would report that the signature verified in this case.  So I don't think that there is any security issue here.

Yes, in theory, we could include a consensus call question giving people this option as well, but unless support for it shows up on the list, it would just seem (to me) to unnecessarily complicate the questions we're asking people by including an option that's not actually viable from a complexity, interoperability, or security point of view.

                                                            -- Mike

From: Jim Schaad [mailto:[e-mail]]
Sent: Wednesday, December 26, 2012 2:03 PM
To: Mike Jones; [e-mail]<mailto:[e-mail]>
Subject: RE: [jose] Whether implementations must understand all JOSE header fields

This does not include my recommended approach which is to punt the issue of required headers to the application rather than JOSE.

Jim


From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]] On Behalf Of Mike Jones
Sent: Friday, December 14, 2012 10:17 AM
To: [e-mail]<mailto:[e-mail]>
Subject: [jose] Whether implementations must understand all JOSE header fields


Currently the JOSE specs contain the following language declaring that all header (and some other) fields must be understood:  "Implementations MUST understand the entire contents of the header; otherwise, the ... MUST be rejected."  There's currently an open issue about whether all header fields should be required to be understood by implementations using them or whether to relax this requirement in some circumstances.  We would like the working group to consider the following potential resolutions to the issue.



1.  Maintain the current requirement that all header fields must be understood by implementations using them.



PRO:  This is a simple rule designed to result in secure implementations.  Representations of existing JWS & JWE objects are unchanged.

CON:  Extensibility is limited to cases where all participating parties understand new fields that are introduced.



2A.  Take an alternative approach where, by default, all fields must be understood, but where an explicit list of fields can be designated as "may be ignored if not understood".  For instance, an "ign" (may be ignored) member could be defined whose value is an array listing the member names that may be safely ignored if not understood.  An example using this field in a JWS header is:



    {"alg":"ES256",

     "ign":["notes"],

     "notes":"This signature need not be checked when the moon is full"

    }



(Obviously, the "ign" field MUST NOT contain the value "ign".)



PRO:  This would enable adding non-critical header fields without breaking implementations.  It would maintain the current semantics, while allowing explicit exceptions to be made in cases where the fields are not security-critical and the structure has a meaningful interpretation without them.  Representations of existing JWS & JWE objects are unchanged.

CON:  Requires coordination of field names and may-be-ignored list, adding some implementation complexity.



2B.  Take an alternative approach where instead of one header object, there would be two header objects.  Implementations would be required to understand all fields in the first header object.  Implementations would be allowed to ignore fields in the second header object if not understood.  An example of this approach, adding the second header object after the first in a JWS, would be:



    eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJIUzI1NiJ9

    .

    eyJub3RlcyI6IlRoaXMgc2lnbmF0dXJlIG5lZWQgbm90IGJlIGNoZWNrZWQgd2hlbiB0aGUgbW9vbiBpcyBmdWxsIn0

    .

    eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ

    .

    dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk



The first, third, and fourth fields are from the example JWS in the current spec. The new second field is the base64url encoded representation of the string '{"notes":"This signature need not be checked when the moon is full"}'.



PRO:  May-be-ignored fields are easily distinguishable from must-be-understood fields.

CON:  Implementations must consult/merge two sets of headers to understand the meaning/structure of the object, adding some implementation complexity.  Existing JWS & JWE object representations would become invalid.



After a discussion period, we believe that it would be useful for the chairs to make two formal consensus calls of the following kind:



FIRST POLL:  Should all header fields be critical for implementations to understand?

YES - All header fields must continue to be understood by implementations or the input must be rejected.

NO - A means of listing that specific header fields may be safely ignored should be defined.



SECOND POLL:  Should the result of the first poll be "NO", which syntax would you prefer for designating the header fields that may be ignored if not understood?

A - Define a header field that explicitly lists the fields that may be safely ignored if not understood.

B - Introduce a second header, where implementations must understand all fields in the first but they may ignore not-understood fields in the second.



                                                Thanks for considering these questions,

                                                Richard Barnes, John Bradley, Mike Jones