Re: [apps-discuss] Concise Binary Object Representation (CBOR)

Nico Williams <> Fri, 24 May 2013 18:05 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 89B9011E80BA for <>; Fri, 24 May 2013 11:05:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.848
X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5 tests=[AWL=0.129, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id EKgv+vTj7Q-f for <>; Fri, 24 May 2013 11:05:49 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id AC43311E80A2 for <>; Fri, 24 May 2013 11:05:49 -0700 (PDT)
Received: from (localhost []) by (Postfix) with ESMTP id B60B31DE0C4 for <>; Fri, 24 May 2013 11:05:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed;; h= mime-version:in-reply-to:references:date:message-id:subject:from :to:cc:content-type;; bh=CGgT+Ni8Sr4fPlJzaUv1 ofJ09CY=; b=vbDtOQcvmV4/OxBMe5QsLG3++/q0aHNxefXsC8cjpWQC0Je+sfxJ USmO7NwiRrVF07DIAzLMueVqFjnkIeaQmUk7TNGSr4TndifiY+QoMO07fN/Z06bT mSLYreYUcf7uFQFeSiEAZciePn6hlLQhmo3cE8gdNpLfT3iw/xz5QU0=
Received: from ( []) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: by (Postfix) with ESMTPSA id BCBB91DE0CD for <>; Fri, 24 May 2013 10:53:58 -0700 (PDT)
Received: by with SMTP id hr14so49649wib.9 for <>; Fri, 24 May 2013 10:53:57 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gbCJd4pU3KZIuNHX0zwVVrIB75ImkCxpMRRudbKLn/U=; b=Qd62zwZ6OUZ3B6dro907Uoy3u6GqTXc7FghqhiVOvZcTD3en9YhI+YOvZsQJgMVcFe Eaze8rvbfBWT0QMAJ292LlsmBr3XuhHNfKZ2fClaegY9dlqK9iWqio0N4OTJuj8y/peg CI4Jmf1wjvCMBYwQyNZZCGq32wfIY41VX2/ZFIcLHL/Y2Qm4TZIJ8QBNiw083Wx8A4nf OvKa2ci0UzBfN3/qr+IDR4PUDmjH1X103/z9R6o4LJ0ipRWS214uAHdMeb3lAgc9NlCe g1qCP2ulzgo7jNyzQmXcFjfn93b65WK+26XWR6rBVYWdVAyeWe/uM3B8r45/2jaq+mKe 1B5w==
MIME-Version: 1.0
X-Received: by with SMTP id w3mr142924wia.51.1369418037121; Fri, 24 May 2013 10:53:57 -0700 (PDT)
Received: by with HTTP; Fri, 24 May 2013 10:53:56 -0700 (PDT)
In-Reply-To: <>
References: <> <> <>
Date: Fri, 24 May 2013 12:53:56 -0500
Message-ID: <>
From: Nico Williams <>
To: Phillip Hallam-Baker <>
Content-Type: text/plain; charset="UTF-8"
Cc: " Discuss" <>, Paul Hoffman <>
Subject: Re: [apps-discuss] Concise Binary Object Representation (CBOR)
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 24 May 2013 18:05:54 -0000

On Fri, May 24, 2013 at 12:14 PM, Phillip Hallam-Baker <> wrote:
> On Fri, May 24, 2013 at 1:02 PM, Joe Hildebrand (jhildebr)
> <> wrote:
>> >My experience of coding ASN.1 DER (in C and javascript) leads me to
>> >reject any counted scheme as a non starter.
>> I've read the whole thread, and didn't see adequate justification for that
>> worldview.  Could you reiterate it, please?
> If you have a counter scheme it is necessary to have the whole data
> structure in memory to serialize it. Many network applications require the
> use of an intermediary with a fixed buffer length.
> Now ASN.1 BER definite length encoding makes matters even worse by
> specifying lengths in octets rather than the number of entries in an object
> but the same objection applies. And DER encoding is just plain insane as the
> lengths of the lengths also varies.

Yes, but note that CBOR does much better than DER: it counts elements
(of arrays, of objects), not bytes (except for strings, which are

It might be nice to have an array/object of indefinite length encoding
where the end is indicated by a special type whose only purpose is to
mark the end of the array/object (CER has this).  But don't try to use
this for strings: you end up having to escape the end-of-string marker
-- ugh!

> A very frequent use case for an encoding format is to take a stream of data
> of unknown size and package it in real time. For example I would like to be
> able to take a video stream and encrypt it using JSON encryption and then
> put a digital signature at the end.

Note BTW that JSON *is* amenable to online encoding (and decoding).
Binary encodings should keep this feature.

> BSON does not do that quite like I would like but I can fake the same effect
> by using an array of Binary rather than one Binary chunk and the same for
> strings.

I think that's as well as you can expect to do without having to
resort to escaping binary/string characters.

Anyways, +1.