Re: [core] SenML JSON syntax (with multiple base objects)

Cullen Jennings <fluffy@iii.ca> Tue, 27 October 2015 06:06 UTC

Return-Path: <fluffy@iii.ca>
X-Original-To: core@ietfa.amsl.com
Delivered-To: core@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0F61B1B35DE for <core@ietfa.amsl.com>; Mon, 26 Oct 2015 23:06:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.001
X-Spam-Level:
X-Spam-Status: No, score=-0.001 tagged_above=-999 required=5 tests=[BAYES_40=-0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sTu9SrxuRlbP for <core@ietfa.amsl.com>; Mon, 26 Oct 2015 23:06:39 -0700 (PDT)
Received: from smtp109.ord1c.emailsrvr.com (smtp109.ord1c.emailsrvr.com [108.166.43.109]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A41CF1B35DD for <core@ietf.org>; Mon, 26 Oct 2015 23:06:39 -0700 (PDT)
Received: from smtp14.relay.ord1c.emailsrvr.com (localhost.localdomain [127.0.0.1]) by smtp14.relay.ord1c.emailsrvr.com (SMTP Server) with ESMTP id ED7C63800F1; Tue, 27 Oct 2015 02:06:38 -0400 (EDT)
Received: by smtp14.relay.ord1c.emailsrvr.com (Authenticated sender: fluffy-AT-iii.ca) with ESMTPSA id 1CDFA3801F3; Tue, 27 Oct 2015 02:06:32 -0400 (EDT)
X-Sender-Id: fluffy@iii.ca
Received: from tky-vpn-client-230-5.cisco.com ([UNAVAILABLE]. [64.104.44.108]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:587 (trex/5.4.2); Tue, 27 Oct 2015 06:06:38 GMT
Content-Type: multipart/alternative; boundary="Apple-Mail=_C5F854C4-77B2-4280-948B-48742A9D6742"
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
From: Cullen Jennings <fluffy@iii.ca>
In-Reply-To: <956423E0-BA70-4F4F-B0CD-6423203BC2BD@gmail.com>
Date: Tue, 27 Oct 2015 15:08:18 +0900
Message-Id: <3CEB5B46-504F-4F4C-B3B8-294625FAF6EA@iii.ca>
References: <58C4BBDD-895C-4946-983A-405C6E5B760D@ericsson.com> <DAFB7C72-CF08-438E-B681-10918717A5D5@gmail.com> <20151020184630.GI5451@hephaistos.amsuess.com> <956423E0-BA70-4F4F-B0CD-6423203BC2BD@gmail.com>
To: core <core@ietf.org>
X-Mailer: Apple Mail (2.2104)
Archived-At: <http://mailarchive.ietf.org/arch/msg/core/Vgpk5etQ2EDApa_4HzYl2C-8dJY>
Cc: "draft-jennings-core-senml@tools.ietf.org" <draft-jennings-core-senml@tools.ietf.org>, Christian Amsüss <c.amsuess@energyharvesting.at>
Subject: Re: [core] SenML JSON syntax (with multiple base objects)
X-BeenThere: core@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "Constrained RESTful Environments \(CoRE\) Working Group list" <core.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/core>, <mailto:core-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/core/>
List-Post: <mailto:core@ietf.org>
List-Help: <mailto:core-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/core>, <mailto:core-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 27 Oct 2015 06:06:43 -0000

I’ve been toying with the idea that there is only one type of object and there is an array of theses. If a base name is found in the the object, it applies to all the values in that object plus any later objects in the array until something else overrides that base name. 

So the JSON might look like 


[
  { “bn”: ”/collection1/“ },

  {“n”: “item1”, “sv”, “value1”},
  {“n”: “item2”, “sv”, “value2”},

  { “bn”: ”/collection2/“ },

  {“n”: “thing1”, “sv”, “value1”},
  {“n”: “thing2”, “sv”, “value2”}
]


Which could be the same thing as say 

[
  { “bn”: ”/collection1/“ , “n”: “item1”, “sv”, “value1”},
  {“n”: “item2”, “sv”, “value2”},

  { “bn”: ”/collection2/“, “n”: “thing1”, “sv”, “value1”},
  { “bn”: ”/collection2/“, “n”: “thing2”, “sv”, “value2”}
]


I’m thinking implementation would 
> On Oct 21, 2015, at 4:52 AM, Michael Koster <michaeljohnkoster@gmail.com> wrote:
> 
> Thanks for the reply! My comments below.
> 
> Michael
> 
> On Oct 20, 2015, at 11:46 AM, Christian Amsüss <c.amsuess@energyharvesting.at> wrote:
> 
>> Hello Michael,
>> 
>> On Tue, Oct 20, 2015 at 11:06:24AM -0700, Michael Koster wrote:
>>> Is the proposed format simpler because it removes the need to have an
>>> “e” tag and instead associates a data object with a base object by
>>> position in the array?
>> 
>> the main advantage of the [{base}, [elements]] serialization is that
>> constrained devices have a chance of streaming decoding. With the base
>> and element data contained in an object, it is impractical to prescribe
>> a sequence of serialization (as it precludes the use of common JSON
>> serializers whose data structures don't know order inside an object).
>> 
>> There has been some discussion at [1] and [2], which has led to the the
>> currently proposed syntax.
>> 
>>> I think having base parameters and data array wrapped together in an
>>> object has some advantages, particularly when the object represents a
>>> set of resources as opposed to a sequence of measurements from a base
>>> resource. In this case the base name is the object or collection name
>>> and the array represents the resources in the collection.
>> 
>> Granted, the grouping between the base name and the elements is not that
>> (visually) tight any more, but can still be expressed with the same base
>> name mechanisms. How is the multiple-resources case different from the
>> multiple-timestamps case here?
> 
> It’s more than a simple visual relationship. I’m used to JSON tools that create an in-memory data structure that conforms to the JSON serialization. With the “old” SenML model, the elements of the object identified by “bn” are rendered as an array within the element identified by “bn” and tagged by “e”. 
> 
> The new construct more than just enables streaming, it forces serial interpretation, i.e. it *requires* streaming. I need to transform the in-memory representation if I want it to look the same as the familiar one. Not that I can’t do that, but to illustrate that it’s more than just a visual preference thing. It’s a matter of “impedance match” between the software machine and the in-memory representation I get from a JSON destringifier.
>> 
>>> Additionally, this format can be extended using the same mechanism by
>>> adding another tag for another class of element. For example, let’s
>>> say I wanted to extend senml to add descriptive metadata in the form
>>> of RFC6690 hyperlinks encoded in link-format+json. I could easily add
>>> a “l” element extension for links and create a new content-format
>>> which can be processed and understood as a simple extension of SenML:
>> 
>> That indeed has not come up in the previous discussion, and would not be
>> easily feasible with the new syntax.
>> 
>> In my opinion, it raises the question of how generic SenML should
>> attempt to be. My personal view of it is that SenML is a way of
>> encapsulating several resource representations (be they of different
>> points in time or different resource) in a single message. With that in
>> mind, maybe the following would work for you (rephrasing your example
>> into senml-02 syntax, with comments):
>> 
> SenML is already being used to represent simple collections in CoRE Interfaces, OMA LWM2M, and OIC. Whether to have it be extensible and evolvable or not is certainly a tradeoff against complexity and stream processing ability. I would lean toward evolvability. 
> 
> Would it make sense to create a new content-format that optimizes for streaming processing?
> 
>> [
>>   { “bn”: ”/collection1/“ }
>>   [ 
>>     /* the sv values are identical to what the client would receive
>>        when GETting the respective /collection1/item as text/plain */
>>     {“n”: “item1”, “sv”, “value1”},
>>     {“n”: “item2”, “sv”, “value2”},
>>     {“n”: “item3”, “sv”, “value3”},
>>     {“n”: “item4”, “sv”, “value4”},
>> 
>>     /* the "object value" is what the client would receive when
>>        GETting /collection1/ with mime type
>>        application/link-format+json */
>>     {"n": "", "ov": [
>>       {“href”: “item1”, “if”, “core.s”, “rt”: “type1”, “ct”: “50"},
>>       {“href”: “item2”, “if”, “core.s”, “rt”: “type2”, “ct”: “50"},
>>       {“href”: “item3”, “if”, “core.s”, “rt”: “type3”, “ct”: “50"},
>>       {“href”: “item4”, “if”, “core.s”, “rt”: “type4”, “ct”: “50"}
>>     ]}
>>   ]
>> ]
>> 
> 
> I was going to ask for “ov” for the ability to represent items that are themselves objects, so that’s a very useful feature for collections. However, representation of the links in a collection as a collection mapped to one of the items does not lead to a natural in-memory representation either. 
> 
>> That would need another data type in addition to string-, (numeric)- and
>> boolean value ("sv", "v", "bv") and thus allow encapsulating any other
>> JSON based response. Parsers in applications that don't expect "ov"
>> wouldn't even need to get more complicated, as they are already required
>> to ignore key-value pairs they don't understand.
>> 
>> This still doesn't give a way to add arrays in parallel to the previous
>> "e" element, but for this application, it would automatically give
>> streamability also to your "l" list.
>> 
>> What do you think of the above arrangement?
>> 
> I think it’s a substantial compromise in the ability to represent data structure to get streaming processing ability. But I do like the idea of a “ov” element for object values.
> 
> Best regards,
> 
> Michael
> _______________________________________________
> core mailing list
> core@ietf.org
> https://www.ietf.org/mailman/listinfo/core