Re: [apps-discuss] Concise Binary Object Representation (CBOR) -- support streaming

"Manger, James H" <James.H.Manger@team.telstra.com> Thu, 23 May 2013 01:49 UTC

Return-Path: <James.H.Manger@team.telstra.com>
X-Original-To: apps-discuss@ietfa.amsl.com
Delivered-To: apps-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CC70611E8181 for <apps-discuss@ietfa.amsl.com>; Wed, 22 May 2013 18:49:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.325
X-Spam-Level:
X-Spam-Status: No, score=-0.325 tagged_above=-999 required=5 tests=[AWL=0.576, BAYES_00=-2.599, HELO_EQ_AU=0.377, HOST_EQ_AU=0.327, RELAY_IS_203=0.994]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5uSF1Fmo6b+0 for <apps-discuss@ietfa.amsl.com>; Wed, 22 May 2013 18:49:26 -0700 (PDT)
Received: from ipxano.tcif.telstra.com.au (ipxano.tcif.telstra.com.au [203.35.82.200]) by ietfa.amsl.com (Postfix) with ESMTP id F17A911E815B for <apps-discuss@ietf.org>; Wed, 22 May 2013 18:49:23 -0700 (PDT)
X-IronPort-AV: E=Sophos;i="4.87,725,1363093200"; d="scan'208";a="138769329"
Received: from unknown (HELO ipccni.tcif.telstra.com.au) ([10.97.216.208]) by ipoani.tcif.telstra.com.au with ESMTP; 23 May 2013 11:49:20 +1000
X-IronPort-AV: E=McAfee;i="5400,1158,7083"; a="138776811"
Received: from wsmsg3757.srv.dir.telstra.com ([172.49.40.85]) by ipccni.tcif.telstra.com.au with ESMTP; 23 May 2013 11:49:20 +1000
Received: from WSMSG3153V.srv.dir.telstra.com ([172.49.40.159]) by wsmsg3757.srv.dir.telstra.com ([172.49.40.85]) with mapi; Thu, 23 May 2013 11:49:20 +1000
From: "Manger, James H" <James.H.Manger@team.telstra.com>
To: "apps-discuss@ietf.org" <apps-discuss@ietf.org>
Date: Thu, 23 May 2013 11:49:19 +1000
Thread-Topic: [apps-discuss] Concise Binary Object Representation (CBOR) -- support streaming
Thread-Index: Ac5W+5nrcSLuagvXSgiAuBQgwxS2KQAVVUVQ
Message-ID: <255B9BB34FB7D647A506DC292726F6E1151A8ACB34@WSMSG3153V.srv.dir.telstra.com>
References: <61CB1D18-BABC-4C77-93E6-A9E8CDA8326B@vpnc.org>
In-Reply-To: <61CB1D18-BABC-4C77-93E6-A9E8CDA8326B@vpnc.org>
Accept-Language: en-US, en-AU
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US, en-AU
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Subject: Re: [apps-discuss] Concise Binary Object Representation (CBOR) -- support streaming
X-BeenThere: apps-discuss@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <apps-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/apps-discuss>
List-Post: <mailto:apps-discuss@ietf.org>
List-Help: <mailto:apps-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 23 May 2013 01:49:33 -0000

CBOR [draft-bormann-cbor] looks great.

The only big problem I see is that sizes have to be known upfront. You need to know how many bytes, array elements, or map pairs you have BEFORE you start encoding them. This really hinders streaming content. JSON does not have this limitation so CBOR should not introduce it. Removing this limitation would better meet objective 5: "be applicable to both constrained nodes and high-volume applications".

This should not be hard to solve, nor to implement.
For instance define initial bytes for "Array", "Map", "Byte array in parts", and "End". An array is the items between "Array" and "End". A map is an even number of items between "Map" and "End". There would be no need for the 64 initial bytes assigned to known-size versions of arrays and maps.

The 32 initial bytes for known-length byte arrays would remain. The "Byte array in parts" initial byte would be an additional option. Between "Byte array in parts" and "End" can be any number of known-length byte array items.

--
James Manger