Re: [T2TRG] [Asdf] [One Data Model Group] Representation issues and protocol bindings

[David Kemp <[e-mail]>]

On Mon, Aug 31, 2020 at 4:19 PM Carsten Bormann <[e-mail]> wrote:

> I think it is already relatively easy to find the right level of
> abstraction for information that can be expressed in SI units.
> (Units can be specified in SDF 1.0.)
>
> SenML (RFC 8428) got that right, so I think we can follow that example in
> SDF.
> We did have to make a little bit of a compromise (adding scaling prefixes
> in RFC 8798) in areas where the natural units are habitually scaled (there
> aren’t many electricity meters indicating J — it’s always kWh), but even
> here it should be easy to agree on a natural unit between domain experts.
>

 SenML's primary units and secondary (scaled) units did get that right,
with nomenclature and values that make sense to subject matter experts:

{"u":"%RH", "v":20},

{"u":"lon", "v":24.30621},

{"u":"lat", "v":60.07965},

{"n":"temperature","u":"Cel","v":25.2},

{"n":"EGT", "u":"Fahr", "v":1374}


(I made the last one up - the U.S. should have gone metric back in the
1970's, but it didn't.  In this backward part of the world subject
matter experts

might appreciate a familiar secondary unit of temperature.  "F" was
already taken for farad, so ...)


But the problem remains how the ecosystem specifications can relate to the
> converged model if that had to abstract some more.  And that’s where the
> binding comes in.  To make ecosystem experts feel at home, we probably need
> to make this rather accessible.

For information that is a ratio to a reference value (e.g., a lamp an
> undefined metric of which can be set to a value between 0 to 100 %), SenML
> has “/”  (the dimensionless unit).  So we would normally not use
> percentages (or permille, or ppm, …), but a number that is typically
> clamped between 0 and 1.  Of course, most ecosystems will scale this to get
> an integer representation, and that is probably the largest source of
> variation (*).
>

Are there separate problems to be solved?
1) Devices that could but for some reason don't send/receive data in
meaningful units (e.g., uint8 values from 0 .. 255 or 0..100) instead of 38
%RH?
2) Devices for which it is impossible to use absolute units (3.7 volts on a
10v dimmer circuit -> uint8 37 on a scale of 0-100 -> 0,37 dimensionless ->
some level of room illumination that architects and occupational health
people need to measure but consumers don't care about and the controller
can't know)
3) Devices for which meaningful units haven't been defined?


> Also, RFC 8798 is limited to linear scales (scale/offset).  For sound and
> RF power levels we can cheat with dB.  For lighting, the models are more
> complicated (which led to gamma and all that).  If different ecosystems use
> different models here, there may be no straightforward way to do the
> conversions.  So we have to be prepared to express these anyway.
>

A parametric model might be useful for problems 1 and 2.  Linear scales
define min and max values for data (transmitted bits) and information
(subject-meaningful values):

-128  ->  -40.0 Cel
 127 -> 300.0 Cel

or

  0 -> -40.0 Cel
65,536 -> 300.0 Cel

But for SDF that could be generalized to allow calibration points, the way
gradients in an image editor specify at least start and end colors, but may
also add intermediate values.  The domain of the transfer function is
scalar and integer or real, but the range is always real and could be
non-linear and multi-dimensional:

0 -> (0.4, 0.8, 0.25)           min domain value
25 -> (0.5, 0.8, 0.9)           intermediate control point(s)
100 -> (0.0, 0.0, 0.1)         max domain value

After doing linear interpolation to get the range values between control
points, an output function selected from a list could be applied.  The
initial list of output functions might be quite small - I can't think of
any good examples other than linear and log:

1: linear: y = x
2: exponential: y = e^x

The parametric model of a mapping table followed by an optional output
curve might cover 95% of use cases instead of 80%.  Is it simple enough for
an SDF description?

Regards,
Dave