[Cbor] Handling Endianness with CBOR Tagged Arrays

Laurence Lundblade <lgl@island-resort.com> Wed, 08 May 2019 16:37 UTC

Return-Path: <lgl@island-resort.com>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id AEBF812016F for <cbor@ietfa.amsl.com>; Wed, 8 May 2019 09:37:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id OQiGyWI-W9o3 for <cbor@ietfa.amsl.com>; Wed, 8 May 2019 09:37:56 -0700 (PDT)
Received: from p3plsmtpa06-06.prod.phx3.secureserver.net (p3plsmtpa06-06.prod.phx3.secureserver.net []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B950D12015B for <cbor@ietf.org>; Wed, 8 May 2019 09:37:56 -0700 (PDT)
Received: from [] ([]) by :SMTPAUTH: with ESMTPSA id OPZqh1M4hiuL7OPZrhn6bp; Wed, 08 May 2019 09:37:56 -0700
From: Laurence Lundblade <lgl@island-resort.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_94BF20D5-BDA4-4AD2-8F11-741810CC2192"
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Message-Id: <B4B82042-73A6-44D4-9B7D-C64D4B03402E@island-resort.com>
Date: Wed, 8 May 2019 09:37:54 -0700
To: cbor@ietf.org
X-Mailer: Apple Mail (2.3445.9.1)
X-CMAE-Envelope: MS4wfOITa8dxsbvxzOxdPSv4/m3cJTS4q1FHStyXp5qtiJAwwlvlbfnxMd/+2BVYIQxM2auU1Aa9EFU2zm173VlYRL4p4QflX49Mw8S/PFXphApBgPFcjOCD twcA9+6HDQpNUaIl0vm0+KkK6zYMofVeLjyTS1lxuOi++ZKrV6yCRmwo
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/YLQwpUGnn8rEWbR5edWbbzHWe0w>
Subject: [Cbor] Handling Endianness with CBOR Tagged Arrays
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 May 2019 16:37:59 -0000

Was thinking about how to ensure interoperability with tagged arrays given the divergence from requiring network byte order. I came up with this text that might go into some document.


Handling Endianness with CBOR Tagged Arrays

Every CBOR protocol using tagged arrays must specify how endianness is handled or there will be interoperability problems. The following are five options, that last of which is not recommended.

1) Default to little-endian
By not specifying anything, the default of little-endian is taken.

This is a pretty good option for many protocols as such a large proportion of CPUs today are little-endian.

2) Specify the endianness
Here the protocol explicitly defines whether big or little endian is to be used. This is a static definition for every implementation of the protocol and it is incorrect to use other than what is defined.

This is only necessary when big-endian is the only endianness allowed. Given how few CPUs support big endian, use of this is likely to be rare as it will force the performance and likely memory penalty on the CPUs the are little-endian.

3) Asymmetric 
Here one side of the protocol, typically the constrained side, sends whatever it wants. It tells the other side what it is going to send and receive. The other side, typically the server, must be able to receive and send both big and little-endian.

The constrained side can tell the server side by something explicit in the protocol, or if the flow of the protocol has the constrained side sending the first message with a tagged array, that can be the indicator.

This is the best option to accommodate constrained devices and keep code size down.

4) Step-up
Both sides are required to implement a base endianness specified by the particular CBOR protocol. Then, there is a negotiation to step up to a preferred endianness if both support it and prefer it. 

This is may not keep code size down, but it can enhance performance as endian conversion can be avoided. It may also reduce memory use, especially on CPUs that can performed unaligned access. The arrays can be read and written directly from memory to the network.

5) Negotiation
One side of the protocol can send a list of which endianness it supports. The other side picks the one in the list it prefers and tells the first side. Note that this can fail to interop and is not recommended.

Basically, both sides have to agree to implement one specific endianness, or one side has to agree to implement both. Interop can fail if both sides just do what they want.