Re: [jose] My quest to learn how to create SubjectPublicKeyInfo values from scratch

[Mike Jones <[e-mail]>]

Hi Stephen,

Thanks for taking the time to put together the resources cited below.  I spent about 90 minutes looking at all of them on the plane to Dallas and probably another 60 minutes after arriving.  Here's my thoughts on which of them would be useful to the task of creating SubjectPublicKeyInfo values from scratch and what gaps remain.

[0] is a POSIX shell program that generates SPKI Fingerprints from PEM-encoded certificates. This isn't applicable, since if all you have is a JWK, you don't have a PEM-encoded certificate.

[1] also counts on already having the ASN.1 representation in hand, so isn't applicable.  I'll note that https://tools.ietf.org/html/rfc5280#section-4.2.1.2 recommends two different key identifier algorithms, so we're already in a situation where important RFCs define more than one recommended way to generate a Key ID value.  (And that's without taking into account that different hash algorithms and truncated hashes might be used.)

[2] is better, in that it provides actionable instructions for creating SubjectPublicKeyInfo values such as https://tools.ietf.org/html/rfc3279#section-2.3.1, but they're actionable only if you have an ASN.1 encoder available.  Half way there.

[3] is also half-way there, since it's actionable *if* you have an ASN.1 encoder.

[4] Most of the first page of search results here don't provide actionable information for the task.  The exception is the BouncyCastle page with constructor functions, such as "SubjectPublicKeyInfo(AlgorithmIdentifier algId, byte[] publicKey)".  Again this isn't from scratch, but if you have it, you could probably work out how to convert a JWK to a publicKey byte array and go from there (once you also find "new RSAPublicKeyStructuremodulus, exponent).getDERObject())" to do the ASN.1 encoding for you.)  All these search results with actionable content counted on having an ASN.1 library.

[5] was similar to [4], except the partially-actionable information found always counted on having openssl available.

[6] The first 4 pages displayed required openssl, the next 4 were key pinning references, the 9th required openssl, and the 10th was XMLDSIG!

[7] This was the first one that really did the job, for the cases of 1024 and 2048 bit RSA keys with the high bit set and an exponent value of 65537.

[8] also doesn't require ASN.1 or X.509 but like [7] is limited to RSA keys.

[9] implements [10] but I couldn't find the string SubjectPublicKeyInfo info at least in the PHP and Python code I browsed, nor could I find code like that in [7] that really does the job.  I could have missed it.

In summary, I think the core thing we're talking about is really this assertion in [10]:  "formatting any public key as a SubjectPublicKeyInfo is relatively straightforward and well supported by libraries".  The "well supported by libraries" is refuted for some common development environments by the data in Nat's message http://www.ietf.org/mail-archive/web/jose/current/msg04958.html - especially that at present SPKI support in PHP deployments stands at about 0.7%.  So that leaves us with the "relatively straightforward" assertion.

Yes, the code in [7] and Richard's similar code meet the "relatively straightforward" test for the case of a subset of RSA keys.  It doesn't support other key types or some other RSA keys.  That's the primary gap remaining that I see in what's cited above.

Don't get me wrong - I think that hashes of SPKI values are a fine thing for many uses cases.  I also think it's not the only reasonable hash input to consider using.  Heck, [1], which you were an author of, recommends "two common methods".  But despite having put a few more hours into this and tried to genuinely consider all the content in your references, I still haven't seen any place that provides an actionable description of how to create SubjectPublicKeyInfo values from scratch for even all the common RSA and EC key representations in use today.  (I'll repeat that I think that writing this down in once place as a spec would be a valuable contribution, if anyone wants to take this on.  The lack of this simple spec is the real gap in more easily enabling this option, IMO.)

You can dislike it because it uses a different hash input, but at least draft-ietf-jose-jwk-thumbprint does write down how to easily do this for all currently defined JWKs in one place.  Like hashing SPKI values, it's also a fine thing for many use cases.

It's not "I want to do what I thought of first."  It's "I want to give developers semantically sound choices they will gravitate to and actually build."  This is the same reason we did JOSE when we already had CMS and XMLDSIG.

I'll close by agreeing with a statement made in http://tools.ietf.org/html/rfc6920#section-2 - "Hashes are what count".  This is the "semantically sound" thing I was referring to above.  Nat and I and several others are trying to increase the use of hash-based key IDs by putting draft-ietf-jose-jwk-thumbprint out there as an easy option for developers who are comfortable with JSON and who are already using the JWK JSON-based key representation.  The benefit to be gained by more developers having the perception that it's easy to use hash-based key IDs is greater than the benefit of trying to make everyone do it the same way, and have some choose not to do it at all, because right-or-wrong, it just seems too hard to them.  Perception matters.

Anyway, I really have enjoyed this learning exercise and appreciate everyone who provided data in response to this thread.  Good stuff.

I'll look forward to seeing many of you in not too many hours.  Hopefully we can continue this discussion in person.  I know that I'm learning things, anyway, which is always good.

				Cheers,
				-- Mike

-----Original Message-----
From: Stephen Farrell [mailto:[e-mail]] 
Sent: Wednesday, March 11, 2015 7:42 AM
To: Mike Jones
Cc: [e-mail]; Nat Sakimura
Subject: Re: [jose] My quest to learn how to create SubjectPublicKeyInfo values from scratch


Hi Mike,

Simplest is [0] as used in public key pinning for web servers. (That should pop out as an RFC any time now
btw.) I really doubt any claim that that there's some magic needed to make this work as those two lines of script show.

But given you wanted to learn, and not just get stuff done, it's a pity you didn't start from RFC5280, [1] and RFCs 3279 [2] and 5480. [3] Lots of pages there it's true, but actually only very few need to be read if one only cares about SPKI.

Or, maybe just search for the thing you're after [4] and you'll see a bunch of fine information, including howto in the search is even better. [5] Or, if you want code examples those are there too. [6]

I have to admit to being more than surprised that 5 hours of effort didn't throw up any of that.

But if, after that, you're still desperate, then you could look at code I wrote, (you would need to be desperate to try learn from my crappy code:-) [7] being an example of doing this for RSA in about a dozen JS LOC without any
ASN.1 support using the Stanford JS library, and [8] being openssl 'C' code. Or the netinf code [9] implements
RFC6920 [10] with implementations of what you need in other languages like php, python and ruby as well, even clojure if you want to be fancy:-)

Anyway, it took me ~20 minutes to find all those again, and I guess it might take a while to read everything and find the bits you want, but from my POV if someone is developing a generic library for this kind of thing, they really should understand all this already, (or I don't want them writing crypto code on which I depend) or else if all that's needed a quick bit of code for say a client that emits a key id, then the stackoverflow approach of copying from examples should be fine.

Either way, there is IMO not even a scintilla of credibility to any claim that this is super complex or anything like it.

I think I'd summarise the real argument against SPKI here as
being: "I want to do what I thought of first." And of course since that's not a very good argument, further discussion seems to dive into even worse argument, such as this being too difficult, taking hours or being nasty-old-ASN.1 etc.

Cheers,
S.

[0] https://tools.ietf.org/html/draft-ietf-websec-key-pinning-21#appendix-A
[1] https://tools.ietf.org/html/rfc5280
[2] https://tools.ietf.org/html/rfc3279
[3] https://tools.ietf.org/html/rfc5480
[4]
https://www.google.ie/search?q=subjectpublickeyinfo&sa=G&gbv=1&sei=aC4AVdP1OcHP7gaO9oHoBw
[5]
https://www.google.ie/search?q=subjectpublickeyinfo+howto&btnG=Search&gbv=1
[6] https://www.google.ie/search?q=sha256+spki+code&btnG=Search&gbv=1
[7] http://sourceforge.net/p/hoba/code/ci/master/tree/js/hoba-gen-key.js#l60
[8] http://sourceforge.net/p/hoba/code/ci/master/tree/lib/hoba-crypt.cc#l74
[9] http://sourceforge.net/p/netinf/code/ci/default/tree/
[10] http://tools.ietf.org/html/rfc6920

On 11/03/15 05:16, Mike Jones wrote:
> I've always loved learning new things, so I decided yesterday to try 
> to learn first-hand how to write code that emitted X.509 
> SubjectPublicKeyInfo (SPKI) values from scratch.  By "from scratch", I 
> mean using development tools without built-in X.509 or ASN.1 support.
> 
> I took this on because of Stephen's suggestion 
> http://www.ietf.org/mail-archive/web/jose/current/msg04954.html that 
> people could just hash the SPKI values to create a key thumbprint.
> Given I'd helped create the JSON-based hash input described in 
> http://tools.ietf.org/html/draft-ietf-jose-jwk-thumbprint-03, I wanted 
> to give his alternative suggestion a fair shake (and learn some new 
> things along the way).  This admittedly stream-of-consciousness and 
> overly long message describes my expedition to date...
> 
> Thus far, I've spent 5 hours trying to learn to do this.  I spent 
> about the first two hours searching for examples of creating the bytes 
> of X.509 certificates or SubjectPublicKeyInfo values without using 
> ASN.1 and/or X.509 libraries.  I failed.
> 
> Next, I tried to read the authoritative reference for what's in the 
> SPKI field - the X.509 spec.  Unfortunately, 
> http://www.itu.int/rec/T-REC-X.509/en told me "This text was produced 
> through a joint activity with ISO and IEC. According to the agreement 
> with our partners, this document is only available through payment."
> Since most developers would stop at that point, I did too.
> 
> After that, I changed tacks and tried to find examples of sample 
> certificates with commentary on what all the values mean - the kind of 
> info developers would want when coding this.  I had better luck with 
> that.  After about another hour of Web searching, I found this really 
> useful example: http://tools.ietf.org/html/rfc7250#appendix-A.
> I also found this one:
> http://www.jensign.com/JavaScience/dotnet/JKeyNet/index.html.  Going 
> through them byte-by-byte enabled me to reverse engineer some of the
> ASN.1 and X.509 constructs used.
> 
> Things I learned by looking at these 1024-bit RSA public key 
> representations included:
> 
> *        ASN.1 uses byte-aligned Tag-Length-Value encodings.
> 
> *        The tags for SEQUENCE, OID, NULL, BIT STRING, and INTEGER
> are respectively 0x30, 0x06, 0x05, 0x03, and 0x02.
> 
> *        These Length values are encoded as follows:
> 
> o   159 - 0x81 0x9f
> 
> o   9 - 0x09
> 
> o   0 - 0x00
> 
> *        The OID 1.2.840.113549.1.1.1 is encoded in 9 bytes as 0x2a
> 0x86 0x48 0x86 0xf7 0x0d 0x01 0x01 0x01.
> 
> *        The OID is followed by an ASN.1 NULL - 0x05 0x00.
> 
> *        The RSA Key is represented as an encapsulated bit field.
> 
> *        There is an apparently unused zero byte (the 22nd byte of
> the SPKI field in the RFC 7250 example) as the first byte of this bit 
> field.
> 
> *        The rest of the bit field contains concatenated
> representations of the modulus and the exponent as ASN.1 INTEGERs.
> 
> *        The 1024 bit modulus is represented in 129 bytes, with the
> first byte being zero.
> 
> This brought me up to hour four.  Next, I went looking for a 2048 bit 
> cert to learn from (especially since JWA requires 2048+ bit RSA keys).  
> I found http://fm4dd.com/openssl/certexamples.htm and chose 
> 2048b-rsa-example-cert.der, from which I also learned:
> 
> *        These length values are encoded as follows:
> 
> o   290 - 0x82 0x01 0x22
> 
> o   257 - 0x82 0x01 0x01
> 
> *        From this, I deduced (possibly incorrectly :)) that if the
> high bit of the first length byte is 0, the remaining 7 bits represent 
> the length, but if the high bit of the first length byte is 1, the 
> remaining 7 bits represent the number of bytes used to represent the 
> actual length.  (Hence the use of 0x81 for representing values in the 
> range 128-255 and the use of 0x82 for representing values in the range 
> 256-32767.)
> 
> *        Length values are represented in big-endian byte order.
> 
> *        The 2048 bit key representation also starts with an
> apparently unused zero byte.
> 
> *        The 2048 bit modulus is represented by 257 bytes, with the
> first byte being zero.
> 
> Things I haven't yet learned that I'd need to know to really write 
> this code:
> 
> *        How are the OIDs in the table at
> http://tools.ietf.org/html/draft-ietf-jose-json-web-algorithms-40#appe
> ndix-A
> represented as ASN.1 OID values?
> 
> *        Are multiple OIDs sometimes present before the ASN.1 NULL,
> and if so, which algorithms require which sets of OIDs in what order?
> 
> *        Is there always the apparently unused zero byte in the key
> representation or if not, when is it present and absent?
> 
> *        Is there always a leading zero byte in the RSA modulus or if
> not, when is it present and absent?
> 
> *        How are elliptic curve keys represented?
> 
> This brought me up to about the fifth hour of my investigation, and I 
> decided to stop and write up my findings to date.  Highlighted 
> versions of the example certificate from RFC 7250 and the SPKI value 
> from fm4dd.com are attached, should any of you want to follow along 
> with my reverse engineering.  Tags are yellow.  Lengths are green.
> OIDs are purple.  The apparently unused byte is red.  Key values are 
> blue.
> 
> I readily admit that I could have easily missed something while 
> searching.  If someone can point me to self-contained descriptions of 
> this information, I'd love to see them!
> 
> ==== CONCLUSIONS ====
> 
> 1.  I think it would be a fine thing to do to write an RFC describing 
> the mapping between key values and their SPKI representations.  This 
> could take the form of a cookbook with entries like "For a 2048 bit 
> RSA key using RSASSA with SHA-256, emit these bytes, filling in slots 
> A and B in the template with the 256 bites of the mantissa and the 3 
> bytes of the exponent".  Based on my searching, I don't think this 
> information exists anywhere in a self-contained form accessible to 
> developers (but I could be wrong, of course).  I'm not going to 
> personally do it, but if any of you want go for it, have at it!
> 
> 2.  If my experience is representative, telling developers to just 
> hash the SPKI representation of a JWK won't be very effective unless 
> they already have X.509 support.  Most will probably give up well 
> before the 5 hours that I've invested to get this this partial 
> understanding of what I'd need to know.  If my experience is 
> representative, draft-ietf-jose-jwk-thumbprint will be much easier to 
> implement for these developers.
> 
> Trying to live in the shoes of developers, -- Mike
> 
> 
> 
> 
> _______________________________________________ jose mailing list 
> [e-mail] https://www.ietf.org/mailman/listinfo/jose
>