Re: [apps-discuss] Concise Binary Object Representation (CBOR)

Phillip Hallam-Baker <> Fri, 24 May 2013 17:14 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 99B0621F967D for <>; Fri, 24 May 2013 10:14:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -3.189
X-Spam-Status: No, score=-3.189 tagged_above=-999 required=5 tests=[AWL=0.409, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id M-Z65Hxp4aPP for <>; Fri, 24 May 2013 10:14:30 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id A34E421F961C for <>; Fri, 24 May 2013 10:14:29 -0700 (PDT)
Received: by with SMTP id x10so4948864lbi.35 for <>; Fri, 24 May 2013 10:14:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=UqRt78CS8vrwVux8oDKsQAcc6+t6rJDPazWyHU8ERDA=; b=AEzXnsFG09FsoN8hTh845Mw03q4l0O6dsqcIrR2iNjYcakFv1G4mBNcpq1JMFP1Nsr heQKgjeCZi3CbWyFFKsAqSVtc1EiZIueVVSdPLu3kOi4ZOMIOkgqKCeQpBFnDfZEc4AI UULyxOYnjx6ugPZOoZPRjJ0hZjZo+EIevRhhhOdrcqd9NyeTAqqfKsPXFNpF1G+gg2fN fcaP1Das/tyaSG6YySaTRLblqaOK+eBL3i5MwZsejlwOWDDH7z2HLjTRJozbUtfI4w74 cpLIAHHdCwdnDJoNGdTGXUs+73+qomHUDgs434ejfGeTNJT/J2yjXLRGlLZY0tmxB76x bQAA==
MIME-Version: 1.0
X-Received: by with SMTP id jt4mr4857844lab.57.1369415668532; Fri, 24 May 2013 10:14:28 -0700 (PDT)
Received: by with HTTP; Fri, 24 May 2013 10:14:28 -0700 (PDT)
In-Reply-To: <>
References: <> <>
Date: Fri, 24 May 2013 13:14:28 -0400
Message-ID: <>
From: Phillip Hallam-Baker <>
To: "Joe Hildebrand (jhildebr)" <>
Content-Type: multipart/alternative; boundary="001a11c35372790cd104dd79eec1"
Cc: Paul Hoffman <>, " Discuss" <>
Subject: Re: [apps-discuss] Concise Binary Object Representation (CBOR)
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 24 May 2013 17:14:34 -0000

On Fri, May 24, 2013 at 1:02 PM, Joe Hildebrand (jhildebr) <> wrote:

> >My experience of coding ASN.1 DER (in C and javascript) leads me to
> >reject any counted scheme as a non starter.
> I've read the whole thread, and didn't see adequate justification for that
> worldview.  Could you reiterate it, please?

If you have a counter scheme it is necessary to have the whole data
structure in memory to serialize it. Many network applications require the
use of an intermediary with a fixed buffer length.

Now ASN.1 BER definite length encoding makes matters even worse by
specifying lengths in octets rather than the number of entries in an object
but the same objection applies. And DER encoding is just plain insane as
the lengths of the lengths also varies.

A very frequent use case for an encoding format is to take a stream of data
of unknown size and package it in real time. For example I would like to be
able to take a video stream and encrypt it using JSON encryption and then
put a digital signature at the end.

BSON does not do that quite like I would like but I can fake the same
effect by using an array of Binary rather than one Binary chunk and the
same for strings.

With a counted representation I can't even fake it. So I won't be able to
use CBOR for my applications which all involve cryptography. I can do what
I need to do in JSON though.

I see no evidence that counts help in the slightest for efficiency. Even
DER encoding does not really help.

Trying to use a packed encoding as an in-memory encoding is just a bad idea.