Re: [jose] Reducing the size of JWS payloads

[Mike Jones <[e-mail]>]

Hi James,



Hmmm - my quick-and-dirty test code used the .NET System.IO.Compression.DeflateStream<http://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream(v=VS.90).aspx> class and CompressionMode.Compress to do the compression.  The documentation says that it implements RFC 1951 DEFLATE.  That being said, I know that one can use different implementations of the same compression algorithm and get different results, depending upon the matches and substitutions used – all of which will uncompress to the original value.  Do you want to swap data to confirm that we’re using the same compression format?



I compressed these bytes:

101 121 74 104 98 71 99 105 79 105 74 83 85 48 69 120 88 122 85 105 76 67 74 108 98 109 77 105 79 105 74 66 77 84 73 52 81 48 74 68 75 48 104 84 77 106 85 50 73 110 48 46 90 109 110 108 113 87 103 106 88 121 113 119 106 114 55 99 88 72 121 115 56 70 55 57 97 110 73 85 73 54 74 50 85 87 100 65 121 82 81 69 99 71 66 85 45 75 80 72 115 101 80 77 57 49 48 95 82 111 84 68 71 117 49 73 87 52 48 68 110 48 100 118 99 100 86 69 106 112 74 99 80 80 78 73 98 122 87 99 77 120 68 105 49 51 49 69 106 101 103 45 98 56 86 105 87 53 89 88 53 111 82 100 89 100 105 82 52 103 77 83 68 68 66 51 109 98 107 73 110 77 78 85 70 84 45 80 75 53 67 117 90 82 110 72 66 50 114 85 75 53 102 104 80 117 70 54 88 70 113 76 76 90 67 71 53 81 95 114 74 109 54 69 118 101 120 45 88 76 99 78 81 65 74 78 97 49 45 54 67 73 85 49 50 87 106 51 109 80 69 120 120 119 57 118 98 110 115 81 68 85 55 66 52 66 102 109 104 100 121 105 102 108 76 65 55 65 101 53 90 71 111 86 82 108 51 65 95 95 121 76 80 88 120 82 106 72 70 104 112 79 101 68 112 95 97 100 120 56 78 121 101 106 70 53 99 122 57 121 68 75 85 76 117 103 78 115 68 77 100 108 72 101 74 81 79 77 71 86 76 89 97 83 90 116 51 75 80 54 97 87 78 83 113 70 65 49 80 72 68 103 45 49 48 99 101 117 84 69 116 113 95 118 80 69 52 45 71 116 101 118 52 78 52 75 52 69 117 100 108 106 52 81 46 65 120 89 56 68 67 116 68 97 71 108 115 98 71 108 106 98 51 82 111 90 81 46 82 120 115 106 103 54 80 73 69 120 99 109 71 83 70 55 76 110 83 69 107 68 113 87 73 75 102 65 119 49 119 90 122 50 88 112 97 98 86 53 80 119 81 115 111 108 75 119 69 97 117 87 89 90 78 69 57 81 49 104 90 74 69 90 46 56 76 88 113 77 100 48 74 76 71 115 120 77 97 66 53 117 111 78 97 77 112 103 55 117 85 87 95 112 52 48 82 108 97 90 72 67 119 77 73 121 122 107



and obtained this output:

0 14 2 241 253 101 121 74 104 98 71 99 105 79 105 74 83 85 48 69 120 88 122 85 105 76 67 74 108 98 109 77 105 79 105 74 66 77 84 73 52 81 48 74 68 75 48 104 84 77 106 85 50 73 110 48 46 90 109 110 108 113 87 103 106 88 121 113 119 106 114 55 99 88 72 121 115 56 70 55 57 97 110 73 85 73 54 74 50 85 87 100 65 121 82 81 69 99 71 66 85 45 75 80 72 115 101 80 77 57 49 48 95 82 111 84 68 71 117 49 73 87 52 48 68 110 48 100 118 99 100 86 69 106 112 74 99 80 80 78 73 98 122 87 99 77 120 68 105 49 51 49 69 106 101 103 45 98 56 86 105 87 53 89 88 53 111 82 100 89 100 105 82 52 103 77 83 68 68 66 51 109 98 107 73 110 77 78 85 70 84 45 80 75 53 67 117 90 82 110 72 66 50 114 85 75 53 102 104 80 117 70 54 88 70 113 76 76 90 67 71 53 81 95 114 74 109 54 69 118 101 120 45 88 76 99 78 81 65 74 78 97 49 45 54 67 73 85 49 50 87 106 51 109 80 69 120 120 119 57 118 98 110 115 81 68 85 55 66 52 66 102 109 104 100 121 105 102 108 76 65 55 65 101 53 90 71 111 86 82 108 51 65 95 95 121 76 80 88 120 82 106 72 70 104 112 79 101 68 112 95 97 100 120 56 78 121 101 106 70 53 99 122 57 121 68 75 85 76 117 103 78 115 68 77 100 108 72 101 74 81 79 77 71 86 76 89 97 83 90 116 51 75 80 54 97 87 78 83 113 70 65 49 80 72 68 103 45 49 48 99 101 117 84 69 116 113 95 118 80 69 52 45 71 116 101 118 52 78 52 75 52 69 117 100 108 106 52 81 46 65 120 89 56 68 67 116 68 97 71 108 115 98 71 108 106 98 51 82 111 90 81 46 82 120 115 106 103 54 80 73 69 120 99 109 71 83 70 55 76 110 83 69 107 68 113 87 73 75 102 65 119 49 119 90 122 50 88 112 97 98 86 53 80 119 81 115 111 108 75 119 69 97 117 87 89 90 78 69 57 81 49 104 90 74 69 90 46 56 76 88 113 77 100 48 74 76 71 115 120 77 97 66 53 117 111 78 97 77 112 103 55 117 85 87 95 112 52 48 82 108 97 90 72 67 119 77 73 121 122 107 1 0 0 255 255



Can you decompress my output and obtain the original text?  Can you send me your compression output?  I’ll then try to uncompress it.



Thanks for double-checking this!



                                                                -- Mike



-----Original Message-----
From: Manger, James H [mailto:[e-mail]]
Sent: Wednesday, December 19, 2012 3:40 PM
To: Mike Jones; [e-mail]
Subject: RE: Reducing the size of JWS payloads



I don't think your calculations are not quite right, Mike.



Deflating the 526 character JWE example from appendix 2 gives 427 bytes, not the 536 you state.



Assuming a JWS header and signature is 365 chars (2048-bit RSA key, as per JWS appendix 2).



JWS_today(JWE) = 365 + B64(526) = 1067

JWS_zip(JWE) = 365 + 15 + B64(427) = 950 = save 117 chars (11%)

JWS_no_b64(JWE) = 365 + 15 + 526 = 906 = save 161 chars (15%)



So "APPLYING DEFLATE TO THE PAYLOAD" does not make things worse, it gives a 11% saving.



These numbers are not that relevant as I agree with John Bradley that an asymmetric signature is more likely to be applied before the encryption.



I suspect JOSE would be better off defining its operations on byte arrays: the first byte array is a JSON object, another byte array is the content; signing covers these byte arrays (with a length) etc. Then leave the base64url encoding to the very last step (after any crypto).



--

James Manger



> -----Original Message-----

> From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]] On Behalf

> Of Mike Jones

> Sent: Wednesday, 19 December 2012 1:26 PM

> To: [e-mail]<mailto:[e-mail]>

> Cc: Dick Hardt

> Subject: [jose] Reducing the size of JWS payloads

>

> I've noticed that more than one person has expressed a desire to

> reduce the size of JWS payloads before signing.  This especially comes

> up when nested encryption and signing is being performed.  This note

> contains a quantitative evaluation of some possible methods of

> reducing JWS payload size and asks for working group input based upon the data.

>

> Dick proposed one method below - have a header parameter to say that

> the payload is already URL-safe and that base64url encoding is not to

> be performed.  Another way that people have proposed is to allow the

> use of the "zip" parameter to compress the JWS payload before

> base64url encoding.  To get some initial data on how the solutions

> compare, I tried both methods using the sample JWE value in

> http://tools.ietf.org/html/draft-ietf-jose-json-web-encryption-

> 07#appendix-A.2 as the JWS payload.

>

> CURRENT SITUATION:  The JWE is 526 characters in length.  Currently,

> when used as a JWS payload, base64url encoding it would increase its

> size to 702 characters - an increase of 33% or 176 characters.

>

> AVOIDING DOUBLE BASE64URL ENCODING:  If we used a header parameter

> "b64":false to indicate that no additional base64url encoding is to be

> performed, the payload would be 176 characters smaller than the

> current situation.  The encoded header size would increase by 16

> characters - the number of characters needed to base64url encode this

> header parameter value and a comma, for a net decrease in size of 160

> characters.  (Yes, parsing the three pieces would be slightly more

> difficult.)

>

> APPLYING DEFLATE TO THE PAYLOAD:  Believe it or not, using DEFLATE on

> this input results in a LARGER output - 536 bytes or a net increase of

> 10 bytes.  If you think about it, this isn't too surprising, as the

> encrypted data should contain no usable predictability/redundancy.

> Base64url encoding these 536 bytes results in a 715 character payload

> - an increase of 189 characters or 36%.  Plus, adding "zip":"DEF" to

> the header adds 16 characters, for a total increase of 29 characters

> over the current situation.  Clearly a suboptimal choice!

>

> CONCLUSION:  Clearly, if we're going to enable reduction of the size

> of JWS payloads, avoiding the double base64url encoding is preferable

> to zip, which actually makes things worse.

>

> QUESTION TO WORKING GROUP:  I'm curious whether people would like to

> see us enable avoiding double base64encoding of JWS payloads when they

> are already URL-safe.  The space savings are significant; they come at

> the cost of the JWS parsing becoming [part before first period . part

> between first and last period . part after last period] rather than

> the current [part before first period . part between first and second

> period . part after second period (with no other periods allowed)].

> Opinions?