Re: [apps-discuss] Concise Binary Object Representation (CBOR)

Phillip Hallam-Baker <> Thu, 23 May 2013 21:30 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 37D4D21F8539 for <>; Thu, 23 May 2013 14:30:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.107
X-Spam-Status: No, score=-3.107 tagged_above=-999 required=5 tests=[AWL=0.491, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id sqlKb7I+DpWf for <>; Thu, 23 May 2013 14:29:48 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 0D48B21F9619 for <>; Thu, 23 May 2013 13:45:42 -0700 (PDT)
Received: by with SMTP id z5so3943858lbh.13 for <>; Thu, 23 May 2013 13:45:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=sMWLedqkKgOcHdUzEn8/0t406i1nkhbB9SRuNF4uM1g=; b=PtIOIsLRUKCBkPmcQOG/Rc4UYZ3jRpYGmbRQJScEBSRmPp9wdG3fu9whjGa6ArvB6P YKd8kMM28bHsP1daZSFhMfq3+vkIS/FK9CajAJNbR5Aek+lVa9K/hyKZ1lCs0kDcGoEK gs0dZwBQ9gYgr3rl3shu5cDWxJItUyz2OWHV3jBcrteGwbG19tF7cov969StjDYIVmHl r8WuAlMD2y42HUA2RQmXM1JWmJFu5jecc9tE/RC8+lQdS1CkKBVMGTjCitLQDmHZ+wmS G9v7rERaLi7KNLVfTq+7bdd1xv9//OZZgmmPTPtF/fWOgI4lCJIEoRneKV4FRQuQ/x2d 4ySw==
MIME-Version: 1.0
X-Received: by with SMTP id w20mr7426476laz.0.1369341941915; Thu, 23 May 2013 13:45:41 -0700 (PDT)
Received: by with HTTP; Thu, 23 May 2013 13:45:41 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <>
Date: Thu, 23 May 2013 16:45:41 -0400
Message-ID: <>
From: Phillip Hallam-Baker <>
To: Nico Williams <>
Content-Type: multipart/alternative; boundary="001a11c237c40627ed04dd68c452"
Cc: Paul Hoffman <>, General discussion of application-layer protocols <>
Subject: Re: [apps-discuss] Concise Binary Object Representation (CBOR)
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 23 May 2013 21:30:04 -0000

On Thu, May 23, 2013 at 4:07 PM, Nico Williams <>wrote:

> On Thu, May 23, 2013 at 2:42 PM, Phillip Hallam-Baker <>
> wrote:
> > One of the main reasons that JSON and XML encodings are attractive is
> that
> > you can look at the wire format and make sense of them. ASN.1 BER
> encoding
> > can also be read with an appropriate dump tool. It has to be possible to
> do
> > the same with Binary JSON.
> Well, they make some sense.  Documentation really helps though :)
> (And BER... can be dumped, but when implicit tagging is used you lose
> plenty of type metadata and have to resort to heuristics.)

I forgot about implicit tagging but that is of course the reason that
implicit tagging in ASN.1 is evil.

> Insisting on a single representation for a number seems much less
> important
> > to me. Whether 7 is represented in one, two, four or eight bytes seems
> to me
> > to be something that can be left to the implementations.
> I agree that it's less important, and I won't insist on it.  It's just
> a mild irritant to not have c14n, but maybe it's just me and I should
> get over it for good.

I think we really have to get over the C14n fetish. It forces us to do
things like sort tags into order before serialization. I have been doing
digital signatures and network security for 20 years now and I have never
once come across a situation where canonicalization would add value except
for situations like DKIM where (1) it is necessary to work round legacy
infrastructure with obnoxious behavior and (2) the ability to verify a
signature is not absolutely critical.

There are only really two cases of interest for authenticating data. The
first is when data is in motion and is passing through gateways and other
intermediaries that might change it. While it is in theory possible that a
gateway might change the data in a way that modifies the bits on the wire
without making semantic changes I have yet to see such a gateway in
practice and what would be the point? Email gateways that perform line
wrapping of text should have been strangled at birth and so should their
XML and JSON equivalents.

The second case is where data is at rest. That is not really the IETF
mandate but the same arguments apply. If you want to authenticate data at
rest then you had better keep the original serialization of the data.

Chunked encoding

One requirement that I have not seen addressed yet is the need to support
chunked encodings so that data can be streamed with a finite buffer space.

If I had a binary coding of JSON available, the number one use case for me
right now would be to package a JSON Signature with the signed data. Or to
use it to create PKCS#7 like structures round an arbitrary MIME content

So I would like to have the ability to specify the length of a text string
or a binary object as a series of chunks, each chunk being preceded by a
length field. It would also be convenient to be able to change the chunk
type from Binary to String in mid stream so as to allow a transcoder that
guessed that the data type was 'base64' based on the first 1000 bytes to
revise that guess when byte 1001 contains a non base64 character.

Null type

Allowing data nulls in a binary format is also very useful. This allows for
padding to align on byte boundaries and can be used to reserve space in a
binary file for pointers and indexes to be added later on.