Re: [pkix] Clarification on "zero" hash value in SigPolicyHash (CAdES)

[Niklas Matthies <[e-mail]>]

Hi Stefano,

Thanks for the information.

I agree that option 3 is somewhat problematic. All-bits-zero can 
conceivably be a valid hash value (I believe for SHA-1 and SHA-2 it's 
actually unknown whether all-bits-zero is reachable), and the standard 
probably shouldn't start disallowing all-bits-zero as a valid hash 
value for all hash algorithms.

I also agree with Peter Gutmann that dropping the feature altogether 
should be a strong consideration, in particular if the original 
motivation can't be identified, and also given that XAdES does not 
have an equivalent feature. Implementations would still be free to 
support older versions of the standard (with an interpretation of 
their choosing) for backwards compatibility, depending on local policy 
and configuration.

Just FYI, I'm maintaining an implementation that (only) supports 
option 2, however it is probably only interoperable with itself 
reagrding that aspect.

Niklas


On Tue 2019-07-16 at 10:43h, Stefan Santesson wrote on pkix:
>Hi Niklas,
>
>I posted your question on the ETSI ESI list and it created some very interesting discussions.
>
>Very briefly there are 3 possible interpretations:
>
>1) Empty. (The value is a empty OCTET STRING of zero bytes)
>2) Zero. (The value is the byte 0x00)
>3) All zeros. (The value is an OCTET STRING of the intended hash value length containing only 0x00 bytes).
>
>
>There are actually some evidence that suggests that the original intent was option 3.
>However, option 3 is the absolute worst from an implementation perspective as it is the hardest to programmatically distinguish from a real hash value.
>
>My conclusion is that the standard is so ambiguous that any receiving implementation should be able to handle all 3 alternatives.
>The standard need to be clarified by ETSI. Until that is done my proposal for sending applications is to use option 1 (empty) because:
>
>1) It is the only option that makes sense and it can be expected as the most probable future resolution
>2) The standard is so ambiguous that validators anyway must expect all options.
>3) This feature is probably not widely implemented. Doing it one way or the other probably has very little impact on interop.
>
>I'll keep you posted if any opinion emerges that materially contradicts this conclusion.
>
>Stefan Santesson
>
>On 2019-07-12, 22:07, "pkix on behalf of Niklas Matthies" <[e-mail] on behalf of [e-mail]> wrote:
>
>    Dear all,
>
>    I hope that someone can shed some light on the definition and origin
>    of the special-case "zero" hash value for SigPolicyHash in CAdES. The
>    main question is what exactly constitutes a "zero" hash value on the
>    ASN.1 level.
>
>    In the most current CAdES specification, ETSI EN 319 122-1 V1.1.1
>    (2016-04), the zero hash value is described as follows (clause
>    5.2.9.1, page 20):
>
>        The sigPolicyHash field shall contain the identifier of the hash
>        algorithm and the hash of the value of the signature policy.
>        The hashValue within the sigPolicyHash may be set to zero to
>        indicate that the policy hash value is not known.
>
>            NOTE: The use of a zero-sigPolicyHash value is to ensure
>            backwards compatibility with earlier versions of ETSI TS 101 733 [1].
>
>    The ASN.1 definition of SigPolicyHash is as follows:
>
>        SignaturePolicyId ::= SEQUENCE {
>            sigPolicyId SigPolicyId,
>            sigPolicyHash SigPolicyHash,
>            sigPolicyQualifiers SEQUENCE SIZE (1..MAX) OF
>                SigPolicyQualifierInfo OPTIONAL
>        }
>
>        SigPolicyHash ::= OtherHashAlgAndValue
>
>        OtherHashAlgAndValue ::= SEQUENCE {
>            hashAlgorithm   AlgorithmIdentifier,
>            hashValue       OtherHashValue }
>
>        OtherHashValue ::= OCTET STRING
>
>    The question is what constitutes a "zero" hash value. A zero-length
>    octet string? Any octet string where all octets are zero? Is the
>    length of the octet string relevant, e.g. can or should it match the
>    length implied by the hash algorithm?
>
>    I spent some time going back the history of the standards to find the
>    "earlier version" relevant for the note about backwards compatibility,
>    however that search was inconclusive.
>
>    All versions of ETSI TS 101 733 from the latest V2.2.1 (2013-04) back
>    to V1.7.4 (2008-07) include the following note, which had a second
>    sentence added:
>
>            NOTE: The use of a zero-sigPolicyHash value is to ensure
>            backwards compatibility with earlier versions of the
>            current document. If sigPolicyHash is zero, then the hash
>            value should not be checked against the calculated hash value
>            of the signature policy.
>
>    Confusingly, this now talks about sigPolicyHash being zero (whatever that
>    would mean), not about the hashValue field within sigPolicyHash.
>
>    The same text appears in RFC 5126 (February 2008).
>
>    The previous version ETSI TS 101 733 V1.7.3 (2007-01) is the first
>    version to talk about a zero hash value, and the note is just one
>    sentence again:
>
>        The hashValue within the sigPolicyHash max [sic] be set to zero to
>        indicate that the policy hash value is not known.
>
>            NOTE: The use of zero policy hash value is to ensure backward
>            compatibility with earlier versions of the current document.
>
>    In addition, the annex states the following:
>
>        Annex K (informative): Changes from the previous version
>
>            • if the hash of the signature policy is unknown, then, by
>            convention, the sigPolicyHash shall be set to all zero
>
>    Here again, the annex talks about sigPolicyHash being set to zero,
>    not the hashValue within sigPolicyHash.
>
>    On the RFC side, the same note first appears in
>    https://tools.ietf.org/html/draft-ietf-smime-cades-02 (May 2007),
>    and the sentence above (without the note) first appears in
>    https://tools.ietf.org/html/draft-ietf-smime-cades-01 (April 2006).
>
>    In the previous ETSI TS 101 733 V1.6.3 (2005-09), the sigPolicyHash
>    field is optional (which is mandatory in all previous and all
>    subsequent versions), and the following note is included:
>
>            NOTE: In the previous version of TS 101 733 (i.e. version 1.5.1)
>            sigPolicyHash was mandatory. Implementations requiring to be
>            backward compatible with version 1.5.1 and previous versions
>            of the current document MUST include SigPolicyHash.
>
>    None of the previous versions, from ETSI TS 101 733 V1.5.1 (2003-12)
>    back to ETSI TS 101 733 V1.2.2 (2000-12) (the earliest version available
>    at https://www.etsi.org/deliver/etsi_ts/101700_101799/101733/), as
>    well as RFC 3126 (September 2001), do mention anything about zero hash
>    values.
>
>    So it remains unclear to which specification backwards compatiblity is
>    intended. The point where the wording about zero hash values was
>    introduced suggests that this may be about V1.6.3, where the
>    sigPolicyHash field was optional. However, it's not clear how a "zero"
>    hash value would achieve compatibility with the case of a missing
>    sigPolicyHash field.
>
>    I would appreciate any clarification on that topic.
>
>    Kind regards,
>    Niklas
>
>    _______________________________________________
>    pkix mailing list
>    [e-mail]
>    https://www.ietf.org/mailman/listinfo/pkix
>
>
>