Re: [core] Designs to resolve streaming issues in SenML

[Michael Koster <[e-mail]>]

Hi Christian,

1.
Thanks for the code example! It seems well optimized indeed.

It also serves to highlight one of the points. 

It’s more than just a matter of me liking my own favorite language (actually library) internal representation of JSON. Most modern libraries produce dictionary and array representations for parsed JSON that work more or less the same way.

The optimizations you recommend require a new string input parser, where I am using the JSON parser that comes with the library. My point was that I will need to either make my own string parser for SenML or make a re-parser with the state machine I described in order to use -02 + but there is another issue anyway:

One of the big advantages of using JSON is connecting the embedded world to the web world. To do this in a low-friction way, it’s good to be able to use well known tools and patterns. Streaming JSON 
2. 
Thanks also for looking into my code. I could easily use the -02 or -03 in the senml processing as you show. What is missing still from everything after -01 is the element tag that I am re-using to indicate links and forms in the document:

{
  "bn":"/3300/1/",
  "e":[
    {"n":"5700","v":"31.3"},
    {"n":"5602","v":"37.1"},
    {"n":"5601","v":"18.3"},
    {"n":"5605"},
    {"n":"5603","v":"0"},
    {"n":"5604","v":"100"},
    {"n":"5750","sv":"Inboard Bearing"}
  ],
  "l":[
    {"href":"","rel":"self","rt":"temperature","u":"C"},
    {"href":"5700","rt":"currentValue","u":"C"},
    {"href":"5602","rt":"maxValue","u":"C"},
    {"href":"5601","rt":"minValue","u":"C"},
    {"href":"5605","rt":"resetMaxMin"},
    {"href":"5603","rt":"minScale","u":"C"},
    {"href":"5604","rt":"maxScale","u":"C"},
    {"href":"5750","rt":"appType"}
  ]
}

I will think about other designs that do not depend on entity tags in the representation; maybe there is a better design for the content format in general. But I would still like to use SenML. Maybe not extend it but embed it in another new format…

In the meantime, I don’t have a good alternative yet.

Best regards,

Michael

PS Can we succinctly describe the use case for ordered serial processing of senml+json elements being important? From reading the draft, it seems like the important bit is saving the buffer memory for large responses. Is it a bigger deal with CBOR encoding or less important. 

Is the most important consideration being able to know the base values ahead of the data? 

Also, where is -03? there doesn’t seem to be a version -03 that you all are talking about on the IETF website:
https://tools.ietf.org/html/draft-jennings-core-senml-04 <https://tools.ietf.org/html/draft-jennings-core-senml-04>


> On Jan 16, 2016, at 5:27 AM, Christian Amsüss <[e-mail]> wrote:
> 
> Hello Cullen,
> 
> On Wed, Jan 13, 2016 at 01:57:02PM +0000, Cullen Jennings (fluffy) wrote:
>> The objects can also have the base values right in the object and the
>> base values apply to all the things in the object and any future
>> objects in the array. 
>> 
>> Appreciate any thoughts people have on pro / cons of the -03 design vs -04 design.
> 
> The -03 draft is parsable with minimum RAM: I've assembled a crude
> demo[1] that shows that given a list of known integer-valued resources
> on a device, it is possible to parse incoming SenML-03 with less than 30
> bytes of state, with support for arbitrary SenML extensions, all legal
> JSON number representations and full URL concatenation semantics.
> 
> With the -04 draft, one would have to allocate a buffer for the "n"
> value, because before the object finishes, the "bn" might change later
> in the same object, and change the context in which it needs to be
> interpreted. This could be remedied by making base and entry updates
> mutually exclusive (arguably a less straight-forward design), but as
> things are, I'd prefer -03.
> 
> Best regards
> Christian
> 
> [1] https://gitlab.com/c.amsuess-energyharvesting/senml-03-streaming-parser-demo/tree/master
> 
> -- 
> Christian Amsüss                      | Energy Harvesting Solutions GmbH
> founder, system architect             | headquarter:
> mailto:[e-mail]  | Arbeitergasse 15, A-4400 Steyr
> tel:[phone]                | http://www.energyharvesting.at/
>                                      | ATU68476614
> _______________________________________________
> core mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/core