Re: [saag] ASN.1 vs. DER Encoding

[Carsten Bormann <[e-mail]>]

On Mar 30, 2019, at 19:43, Christian Huitema <[e-mail]> wrote:
> 
> The TLS syntax appears specifically designed to avoid many of the pitfalls TLV encodings. For example, the syntax defines the fixed encoding length of all integer and length fields, 

… which causes the result to be larger than it needs to be.
Compare https://tools.ietf.org/html/draft-rescorla-tls-ctls-01 or other proposals in this field.

> and uses intermediate octet array encodings for extensions. It is certainly much easier to get right

Yeah, as we all saw in heartbleed. :-)
[Don’t forget to celebrate its fifth discovery birthday tomorrow.]

> than BER or DER.

Oh, that I can concede.

There is no free lunch in this space.

The TLS encoding works by opting for simplicity, but it requires bespoke en-/decoders, which may or may not (more often) be compiled from the TLS “presentation language”.  These are built from a small set of primitives that are reasonably easy to get right even when manually coding.

TLS can do that because it delegates its more complex parts to other formats such as ASN.1 DER.

Since the mid-1990s, there has been a trend in the industry towards encodings that enable generic en-/decoders, handling the lexical level of (de-)serialization.  This, of course, was not new (RFC 713 had it in 1976, and ASN.1 BER decoding is generic on the lowest level, too).  The good thing is that these generic codecs can be hardened and used in a variety of applications; the bad thing is that the hardening does not always happen and gets more complicated with the complexity of the generic codec.  While the focus has been on text codecs (XML, JSON) for a while, binary codecs also exist (CBOR being an example from this decade, patterned on the earlier msgpack).

Binary generic codecs can opt to have redundant bytewise length information (such as BER does), or opt to count items instead of bytes except at the lowest level (as CBOR does).  Apart from causing pain when serially encoding, the ability to exploit this redundancy with inconsistent data is an attacker’s play field.  It also is useful when it is necessary to skip entire subtrees in one go; CBOR requires visiting all nodes on the subtree being skipped (unless the “wrap in byte string” design pattern is employed).

Generic codecs can be intricately tied to an arcane data model that calls for a data description language (as in ASN.1, where the tie is perceived so tight that it triggered this entire exchange, or in XML, which at least had an evolution of data description languages over its lifetime), or try to map to a generic data model that a programmer might want to use directly.  JSON, msgpack, and CBOR share most of one such generic data model and mostly differ on coverage and extensibility at this level.  Data description languages can then help map from that generic data model to the application data model, also possibly validating input data in the process, but are not required (“schemaless” decoding); CDDL is one such data description language.

Grüße, Carsten