RE: Possible ambiguities in encoding of signatures, encrypted keys

[William Whyte <[e-mail]>]

Hi Peter,

> Currently both RFC 2459 and CMS refer to RFC 2313/2437 for the encoding of RSA
> signatures/encrypted data (RFC 2459, 7.2.1; CMS, 12.3.2.1 and 12.2.2 - what I'm
> about to describe applies to other algorithms as well, but I'll stick with RSA
> to keep it simple).  These RFC's make the assumption that the encoded value
> will be of the same length as the modulus, zero-padding the value if required
> (RFC 2437, 7.2.1 and 8.1.1), however when this padding is used the encoded
> value doesn't follow the DER any more.

I'm not sure this is right. The signature is an octet string or a
bit string, not an integer, and it's perfectly legal to have an
OCTET STRING or BIT STRING with leading null bytes. RFC 2313 says:

8.4 Integer-to-octet-string conversion

   The integer encrypted data y shall be converted to an octet string ED
   of length k, the encrypted data. 

and it's the octet string that's encoded.

Cheers,

William