Re: [core] [Last-Call] Last Call: <draft-ietf-core-senml-more-units-02.txt> (Additional Units for SenML) to Proposed Standard

[Hytonen Harri <[e-mail]>]

On Oct 30, 2019, at 17:21, Cullen Jennings <[e-mail]>  wrote:
>  So let's get back to the fundamental issue. Why are we doing this? Some other SDO wants it is not really an answer. Understanding why the other SDO wants it would be a
> good answer. Do the reasons cause more good than interoperability problems that come out of it. I tried to get info from a few other SDO about this but I failed. I’d like to
> hear more about why we should do this. I’m not seeing big value in the saving the floating point calc on the device but perhaps there is a compelling use case where that
> matters that I am not thinking of.

As explained before, it's not matter of saving floating point calculations, but preserving the original precision of the measurement. 

BR,
Harri Hytonen

-----Original Message-----
From: Cullen Jennings <[e-mail]> 
Sent: Wednesday, 30 October, 2019 17:21
To: Ari Keränen <ari.keranen=[e-mail]>; [e-mail]; IETF Crazy <[e-mail]>
Cc: Pete Resnick <[e-mail]>; Hytonen Harri <[e-mail]>; [e-mail]; Gillmore, Matthew <[e-mail]>; [e-mail]; core <[e-mail]>
Subject: Re: [Last-Call] [core] Last Call: <draft-ietf-core-senml-more-units-02.txt> (Additional Units for SenML) to Proposed Standard


If we are going to have add theses secondary scaled units, I do agree with Ari & Carten’s arguments that  having them in a secondary registry that is machine readable is the right way to do it. In fact would be better if IANA also made them available over git but that is a different topic. But here is the problems I see:

Currently interoperability in IoT is a mess and the 20+ SDOs in this space does not help. Us trying to meet the conflicting needs of all theses SDOs will also not help. So let me try and clarity some of the problems with this draft as it stands.

Imagine that today the secondary table has mm but not cm. I write my software to process mm and ship it. Next week cm is added, my software can’t process that, and in many IoT cases, it will take years to be upgraded. At this point, standardizing the table does nothing for anyone and is harmful in that it suggests that things will work together since they are senml but they will not. It’s true that a new unit nothing to do with distance could be added to main table but the way that table is done, that would be rare, and very unlikely my software needs it if my software deals with distance. However, if my software knows and uses m and mm, very likely the cm makes sense for it to use but it can’t. 

The argument for having this second table seems to be an argument to avoiding a single floating point multiple on the IoT decide. Keep in mind this is a device that probably supports EC crypto to send the data. I have not heard any argument that this single multiple is too high a burden on the device. 

The expert review advice for this new table is woefully inadequate for thd broad range of questions the expert will need to answer. For example, will we add “furlongs per fortnight”? why not? Will we add micro-fortnights as a unit? Several RFC use fortnights  (Thank you Dave Oran)

The expert review advise does not cover what is probably the single biggest unit issue with uses. That is dose the type of thing being measured belong in the unit in some cases. For example, is parts per million of oxygen a different unit that parts per million of carbon dioxide? I realize the unit purist would say those are both simply a ratio but if you look at how practitioners use this stuff, theses are totally different units. This draft does not cover how to deal with that. On a side note, I would see way more value of this secondary table if we were clear that we were going to allow both ppm_O2 and ppm_CO2 . However in the past, we argued that was better placed in the name in SENML or in a secondary tag value. 

As a trivial side note  …. the draft is just wrong on how it modifies the expert guidance from the 2nd registry. It says remove rules 4 and 5 from  RFC8428. However the examples we are talking about for this will also probably violate rules 1, 2, 3, 7, 8. I think this would have to be addressed before approving this draft. 

So let's get back to the fundamental issue. Why are we doing this? Some other SDO wants it is not really an answer. Understanding why the other SDO wants it would be a good answer. Do the reasons cause more good than interoperability problems that come out of it. I tried to get info from a few other SDO about this but I failed. I’d like to hear more about why we should do this. I’m not seeing big value in the saving the floating point calc on the device but perhaps there is a compelling use case where that matters that I am not thinking of. 

I think if I understood why people want this, it might change my mind. 


> On Oct 29, 2019, at 2:58 PM, Ari Keränen <ari.keranen=[e-mail]> wrote:
> 
> Hi Pete,
> 
> Carsten replied to many of your questions/comments below, but perhaps one more point to mention is that the additional registry makes it simpler to build a system specification for SenML use where you can simply point that all implementations in that system should use the units from the first registry. Since all the units in the second registry have an equivalent non-scaled-and-shifted version in the first registry, that is rather feasible. Also many implementations are likely to "implement" only a small subset of the units since they would use them to simply check that a value is the kind of value they expected to receive. 
> 
> That said, the most important thing is to get these new units registered as soon as possible, in one or two registries, so that we don't hold back our stakeholders using SenML.
> 
> 
> Cheers,
> Ari
> 
>> On 29. Oct 2019, at 16.16, Pete Resnick <[e-mail]> wrote:
>> 
>> Hi Ari,
>> 
>> What is the upside to having a minimal set of unscaled units when all implementations are going to have to be able to interpret all of the entries from both registries? An implementation can't keep comparisons simple, because it's going to have to deal with all of the new entries anyway. If an implementation is allowed to not implement the ones in the second registry, then that does lead to the interoperability problems Cullen is worried about. Even if if the document says that you MUST be able to interpret values from both registries, the downside is the possibility that implementations might simply ignore the second registry. And once you have the second registry, I don't see that there's any reason to prefer any of the entries in the original registry: You have to be able to handle both, and since everyone has to handle both, there's not reason not to send them either.
>> 
>> Add the scale column to the main registry and use one registry. There's no upside to keeping two, as far as I can tell, and there's definitely the potential for interoperability problems.
>> 
>> pr
>> 
>>> On 28 Oct 2019, at 14:26, Ari Keränen wrote:
>>> 
>>> Hi Pete,
>>> 
>>> I don't have a strong objection for using the same registry for all units, but I think the second registry is a better approach since that allows us to keep the first registry in its original purpose: a minimal set of unscaled units where you would use the numeric part of the value to express the scale. This is still the best approach for many use cases / systems since it allows you to get by with a small amount of identifiers and keeps comparisons across values simple (you don't need to normalize values based on unit's scale before doing comparisons etc.). If we put all the units in the same registry, systems (or their specifications) that would like to have this property would have to state somehow which of the units in the single registry one should use.
>>> 
>>> Also the second registry has the "scale" column that indicates the relation (and enables automatic translation) to the units in the first registry. I guess we could get the same effect by extending the first registry with the scaling field, but at least to me the second registry feels like a cleaner option. And I don't see big downsides for having separate registries for these; both contain still valid SenML units.
>>> 
>>> 
>>> Cheers,
>>> Ari
>>> 
>>>>> On 28. Oct 2019, at 20.58, Pete Resnick <[e-mail]> wrote:
>>>> 
>>>> Matt, Ari, and Harri,
>>>> 
>>>> While you give good support for adding these units, and I am OK with that idea, do you have any objection to adding them to the main registry, which was the other suggestion that Cullen made? I still have not seen a convincing argument not to make these part of the main registry.
>>>> 
>>>> pr
>>>> 
>>>>> On 28 Oct 2019, at 10:40, Gillmore, Matthew wrote:
>>>>> 
>>>>> Hi Cullen,
>>>>> 
>>>>> I am the chair of the IPSO working group in OMA and also support what Ari wrote.  During the last IETF in Montreal, stakeholders from the IPSO working group along with the Core working group met and it was consensus amongst both working groups that the second registry is how we would like to move forward.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Matt
>>>>> 
>>>>> -----Original Message-----
>>>>> From: ietf <[e-mail]> On Behalf Of Ari Keränen
>>>>> Sent: Tuesday, October 22, 2019 6:37 PM
>>>>> To: Cullen Jennings <[e-mail]>
>>>>> Cc: [e-mail]; IETF Crazy <[e-mail]>; core <[e-mail]>; [e-mail]
>>>>> Subject: Re: [core] Last Call: <draft-ietf-core-senml-more-units-02.txt> (Additional Units for SenML) to Proposed Standard
>>>>> 
>>>>> Hi Cullen,
>>>>> 
>>>>> Carsten already replied to some of these issues later in the thread, but let me add some more viewpoints.
>>>>> 
>>>>>>> On 22. Oct 2019, at 17.06, Cullen Jennings <[e-mail]> wrote:
>>>>>> I am strongly opposed to the secondary registry - it will significantly harm interoperability. This could be resolved by not adding the items in the secondary repository or by adding them to the main repository.
>>>>>> 
>>>>>> Having a registry of stuff that may or may not work simply means that we will have less interoperability because what you can not decide what is valid SenML or not.  Sure some of the device vendors making things that produce SenML have thought this was a good idea in the past but I’d like to hear from a bunch of the people that have to process the SenML data that is produced and see if they think it is a good idea.
>>>>> 
>>>>> Both registries would work and their use would result in valid SenML. In a sense the second registry is almost the same as "adding them to the main repository", but having the second registry enables us to give better guidance which units to use/prefer and also provide the translation to the "main registry units".
>>>>> 
>>>>>> The working group discussed this for a long time and was against the items that are being added to the secondary repository.
>>>>>> If there is a large group of people that agree the consensus has changed, then this should simply be put in the main registry. Many people believe that since SenML is not meant to be human readable, having a canonical form of just meters with a floating point number is better than having both km, mm, and god only knows what else.
>>>>> 
>>>>> Yes, the canonical form of "just meters" is still the recommended way in general, but that approach did not work as well as we hoped for all use cases. For example when your sensor value is always in integers of mm/h, it's more convenient and efficient to send integer values than multiples of 2.777e-7. Also in many use cases there is a well-established scaled unit that is used in the industry and forcing to use a different one didn't turn out to be very productive (we can still recommend, though, and that's one of the reasons why we are proposing an additional registry). Finally, legacy models that use non-scaled units but would like to use SenML as the wire format and units registry would be expensive to update to use only non-scaled units.
>>>>> 
>>>>>> I note the secondary repository add mm but not cm. How is this insanity going to work out? How will analytics code trying to process this data know what it can use or not use. If we are going down this path, this is the wrong way to do it. The right way is to define a set of all SI prefixes and say they can be used with any unit.
>>>>> 
>>>>> Any implementation should not try to use anything that is not in the IANA registry. We did also consider to enable all SI prefixes, but concluded that it is still useful to have the smallest reasonable set of units to facilitate interoperability. If there's a real use case that uses a unit with SI prefix that is missing, it can be simply registered. Also not all units in the additional registry would work with just different SI prefix (e.g., min or dBm).
>>>>> 
>>>>> 
>>>>> Cheers,
>>>>> Ari
>>>>> 
>>>>>> 
>>>>>>> On Oct 16, 2019, at 11:42 AM, The IESG <[e-mail]> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> The IESG has received a request from the Constrained RESTful
>>>>>>> Environments WG
>>>>>>> (core) to consider the following document: - 'Additional Units for SenML'
>>>>>>> <draft-ietf-core-senml-more-units-02.txt> as Proposed Standard
>>>>>>> 
>>>>>>> The IESG plans to make a decision in the next few weeks, and solicits
>>>>>>> final comments on this action. Please send substantive comments to
>>>>>>> the [e-mail] mailing lists by 2019-10-30. Exceptionally,
>>>>>>> comments may be sent to [e-mail] instead. In either case, please
>>>>>>> retain the beginning of the Subject line to allow automated sorting.
>>>>>>> 
>>>>>>> Abstract
>>>>>>> 
>>>>>>> 
>>>>>>> The Sensor Measurement Lists (SenML) media type supports the
>>>>>>> indication of units for a quantity represented.  This short document
>>>>>>> registers a number of additional unit names in the IANA registry for
>>>>>>> Units in SenML.  It also defines a registry for secondary units that
>>>>>>> cannot be in SenML's main registry as they are derived by linear
>>>>>>> transformation from units already in that registry; RFC 8428 is
>>>>>>> updated to also accept these units.
> 
> -- 
> last-call mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/last-call