Re: What ASN.1 got right

Phillip Hallam-Baker <phill@hallambaker.com> Tue, 02 March 2021 05:14 UTC

Return-Path: <hallam@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 27D4D3A112F for <ietf@ietfa.amsl.com>; Mon, 1 Mar 2021 21:14:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.399
X-Spam-Level:
X-Spam-Status: No, score=-1.399 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4Sy-eK04TgTF for <ietf@ietfa.amsl.com>; Mon, 1 Mar 2021 21:14:37 -0800 (PST)
Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AB3623A10BF for <ietf@ietf.org>; Mon, 1 Mar 2021 21:14:37 -0800 (PST)
Received: by mail-yb1-f176.google.com with SMTP id x19so19491270ybe.0 for <ietf@ietf.org>; Mon, 01 Mar 2021 21:14:37 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pycHlAJ1BgcS9udJSDbTuU/jJanWeoPUp/U8/Y0QCeQ=; b=ARnYVgjd6tL12MWqq2/GAkPmK8DtXmOkJl8e/T0ec/EuWBNWyBUmrVuFyJ7j6P8mZ3 y42EMy41EFqKRQU7R5i21pyjP2bukQqeSLpEqn+IqPWZ4vrqATFQ6yzGRRI6fgw0F6Vl dUNpOefX5oIEM3P+10x7xRdLE1hHavDLXRsK0chbeZDPLYTizpByFATfLDOzAfDqJO76 9CiZwbPfCVxcyyRZmmVsvJTgOaUtwjP2II0J9Nr3sjsQvLPkVYNMIl6o7jhADHiOlRa1 aka5PBl4RzCEMRX1fm8l1dgKH8UAk07ElFZOAeIlkVFYQDQas/Kd09+XnB8hBPyKfrbo xrow==
X-Gm-Message-State: AOAM532cs6ZZ9EFkN8wZW+zWfUBYeiZHIxY1btadWbWqt1afLtenV6fg fVAgaUecDT+D3TaAA5K2dI/dhWtm0cvDi9ZZXECDN6IAIcs=
X-Google-Smtp-Source: ABdhPJwZlx2Il84u4PfwksKGp150tzqXLYG2gD0gsBv/BQpBuFpfV+oxj+9+h5+9+X+XSkz8abBmiGkxWVwIMa3HIJs=
X-Received: by 2002:a25:2ac3:: with SMTP id q186mr29438663ybq.213.1614662075459; Mon, 01 Mar 2021 21:14:35 -0800 (PST)
MIME-Version: 1.0
References: <20210302010731.GL30153@localhost>
In-Reply-To: <20210302010731.GL30153@localhost>
From: Phillip Hallam-Baker <phill@hallambaker.com>
Date: Tue, 2 Mar 2021 00:14:24 -0500
Message-ID: <CAMm+LwjRm0DFVop12h4o0m2Hthuc-EXKTDJWbJ37gudGr4Wfvw@mail.gmail.com>
Subject: Re: What ASN.1 got right
To: Nico Williams <nico@cryptonector.com>
Cc: IETF Discussion Mailing List <ietf@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000070abed05bc86cf9a"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/D05DK6HRmRgX4aEWILxmeyisvEw>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 02 Mar 2021 05:14:42 -0000

I agree ASN.1 got a lot of things right. And the problem is that if you are
baking a cake, its no longer a cake if there are too many rat droppings in
it.

I have written multiple serializers and deserializers for each of the
commonly used serialization formats, I have used a dozen schema languages
and written another dozen. I created another serialization scheme only last
week as it happens, needed something that works at the packet level that
worked really well with my crypto.

The problem I see with ASN.1 and XML is that the schema languages are
object description languages and thats a different thing from a
serialization format. I have stopped talking about 'objects' on the wire
and now refer to 'records' because I am not doing object broking, this is
message exchange on the wire but they are not events either as I use the
same format for stored data.

So the mistake I see in XML schema and also in most JSON schema proposals
is that they try to capture the full expressive power of the lexical syntax
and I am only interested in exchanging records and mapping them onto my
application data structures. I do not want to have the schema saying 'x is
an integer between 2 and 6'. I want 'x is an integer' and the only time I
want to have more information is if x won't fit into 64 bits. Similarly, I
don't want the schema to say '2 to 6 objects of type foo', the minimum
number of entries in a list is always either 0 or 1 and the maximum is
always 1 or infinity. Nothing else is needed at the schema layer. If you
have detailed data constraints you can always apply them later.



On Mon, Mar 1, 2021 at 8:08 PM Nico Williams <nico@cryptonector.com> wrote:

> As an ASN.1 implementor, I can tell you things I, like many of you, hate
> about ASN.1:
>
>  - ugly syntax (made uglier by having tags as optional lexical elements)
>
>  - ugly OIDs (should have been URNs or URN-like)
>
>  - BER/DER/CER and all TLV encodings in the world (protocol buffers, I'm
>    looking at you too) are awful
>
>  - the fact that the specs were, for a long time, non-free, which led to
>    a dearth of open source tooling that still persists somewhat (though
>    not as bad as it used to be)
>
>  - the fact that for years people hand-rolled their codecs due to the
>    previous item, and so created many bugs
>
>  - X.400/X.500, which are not part of ASN.1, of course, but closely
>    related -- especially X.500 style naming!
>
> Having recently implemented automatic open type decoding handling X.681/
> X.682/X.683 (see RFCs 6025 for some insight, and 5912 for real examples
> that I'm making use of), I can tell you that ASN.1 got some things
> right:
>
>  - rich formalisms ("constraints")
>
>  - separation of syntax and encoding rules
>
> The fact that there are many encoding rules for ASN.1, of almost every
> kind (binary TLV, binaray non-TLV, and textual, including XML- and
> JSON-based rules) proves the last point.  I can't help but see how XDR,
> NDR, and flatbuffers, among many other encodings out there, could easily
> be used with ASN.1.  (E.g., what XDR and NDR call "pointers" are just
> optional fields in ASN.1.)
>
> The fact that one can implement automatic open type decoding using ASN.1
> formalisms and those already published in, e.g., RFC 5912[*] proves the
> utility of those formalisms.  I now get to see certificates and many
> other things in all their "glorious" detail as JSON, including all
> extensions and what not, and that's made possible by these formalisms.
>
> I beg the next person to re-invent this darned wheel to please educate
> themselves as to what came before.  Among many lessons you should learn,
> learn that some things can't be monetized well enough, or at all, or can
> only be monetized in exchange for limiting their adoption, so figure out
> what your goals are, then align pricing and licensing with those.
>
> And since open types appear to be unavoidable, but also always a pain if
> you don't have the metaschema to express the metadata needed to automate
> their handling, don't forget to cover that.
>
> XML, of course, with namespaces and references, has a reasonable
> approach to open types as well.  It's not just ASN.1 that gets that
> right well enough.
>
> Nico
>
> [*] A big thank you to RFC 5912's authors, though sadly one of them has
>     passed.
>
>