Re: [regext] Extension Prefixes, JSON Values, and URI Path Segments

[Mario Loffredo <[e-mail]>]

Hi Tom,

please find my comments embedded.

Il 29/05/2022 06:42, Tom Harrison ha scritto:
> Hi Mario,
>
> On Fri, May 27, 2022 at 12:41:27PM +0200, Mario Loffredo wrote:
>> Think the matter is that even the possible backwards-compatible changes
>> would result in being hardly backwards-compatible.
>>
>> Let te me give an example to make myself clear and move the discussion on a
>> practical perspective.
>>
>> Let's call "ext1" the rdapConformance value signaling the support of
>> "ext1_data" response extension. The response would be:
>>
>> {
>>
>> "rdapConformance" : [ ....., "ext1"],
>>
>> ...
>> "ext1_data" : { ... },
>> ....
>>
>> }
>>
>> Now, let's suppose to add the field "newfield" to "ext1_data" and signal
>> this change by a version update from "ext1" to "ext2". The response should
>> be (in theory):
>>
>> {
>>
>> "rdapConformance" [ ....., "ext2"],
>>
>> ...
>>
>> "ext2_data" : { ... },   // where ext2_data includes all the member of
>> "ext1_data" plus "newfield"
>> ....
>>
>> }
>>
>> To avoid a breakage in the REST contract, the RDAP server should implement a
>> deprecation process ending with the replacement of "ext1_data" with
>> "ext2_data". This means that, for a period of time, both the version of the
>> response extensions should be provided:
>>
>> {
>>
>> "rdapConformance" [ ....., "ext1", "ext2"],
>>
>> ...
>> "ext1_data" : { ... },
>> "ext2_data" : { ... },
>> ....
>>
>> }
>>
>>
>> This situation would result in the following paradox as well: being the
>> introduction of new field in a JSON response widely considered a non
>> breaking change, it should be signaled by a minor version update (e.g.
>> "ext1_1). But, since the final effect would be the replacement of a response
>> field with another,  a major version change should be used instead. In
>> short, every new version of the response extension would imply a major
>> version update ("ext1", "ext2", "ext3", ..) and  a consequent deprecation
>> process the server should support.
> I don't understand the relevance of the scenario given above to the
> approach I set out for handing backwards-compatible changes.  To avoid
> any doubt about the models I was proposing, I'll step through them in
> more detail, per your approach above.
>
>   - Scenario A: add single field (backwards-compatible):
>      - Initial: register extension "ext1", which notes that "ext1_data"
>        will be included in responses, with a single field named
>        "field1":
>
>          "rdapConformance": [ ..., "ext1" ],
>          ...
>          "ext1_data": { "field1": ... }
>
>      - Later: update the existing registration for extension "ext1",
>        noting that some implementations of this extension may now
>        return "field2".  Implementations in accordance with the
>        original version of "ext1" continue working as they do today.
>        Implementations in accordance with the new version of "ext1"
>        return responses like so:
>
>          "rdapConformance": [ ..., "ext1" ],
>          ...
>          "ext1_data": { "field1": ...,
>                         "field2": ... }
>
>      - The presence of "field2" is enough to distinguish the new
>        version from the original version.  Clients written to work with
>        the original version do not change, and clients written to work
>        with the new version are also fine, because they can determine
>        whether the new version has been implemented by way of the
>        presence of "field2".

[ML] Usually, REST-JSON consumers configure JSON libraries to ignore 
unknown fields in deserialization. This just to reduce to prevent from 
breaking changes in responses.

Hence, the presence of "field2" wouldn't be detected unless it was 
managed by the client (e.g. by putting every unknown field on a given map).

>   - Scenario B: add multiple fields (backwards-compatible):
>      - Initial: register extension "ext1", which notes that "ext1_data"
>        will be included in responses, with a single field named
>        "field1".  Subversioning is managed by way of an additional
>        "conformance" field, which is used to note minor
>        (backwards-compatible) updates:
>
>          "rdapConformance": [ ..., "ext1" ],
>          ...
>          "ext1_data": { "conformance": "ext1",
>                         "field1": ... }
>
>      - Later: update the existing registration for extension "ext1",
>        noting that a new "ext1_data.conformance" value of "ext1_1" may
>        be used, implementations of which will return "field2" and
>        "field3" in the "ext1_data" element.  Implementations in
>        accordance with the original version of "ext1" continue working
>        as they do today.  Implementations in accordance with the new
>        version of "ext1" return responses like so:
>
>          "rdapConformance": [ ..., "ext1" ],
>          ...
>          "ext1_data": { "conformance": "ext1_1",
>                         "field1": ...,
>                         "field2": ...,
>                         "field3": ... }
>
>      - The "ext1_data.conformance" field is enough to distinguish the
>        new version from the original version.  Clients written to work
>        with the original version do not change, and clients written to
>        work with the new version are also fine, because they can
>        inspect at the "conformance" field.

[ML]  As a  consequence of this scenario,  every RDAP response extension 
should include a fixed property named "conformance".

If this could be admissible for response extensions defined in the RDAP 
context, it couldn't fit for external specifications.

See JSContact for example. VCard itself is an external specification 
used for defining an RDAP response.

>
> Backwards-incompatible changes continue to require a new top-level
> "rdapConformance" value and a transition process where both
> "ext1_data" and "ext2_data" (or their equivalents) are returned, per
> the example you gave above.

[ML]  Both Scenario A and B seem to me requiring some additional effort 
from implementers. Can't understand why we should make more difficult 
what could be managed straightforwardly.

Think we should accomplish the solution requiring the lowest (to zero) 
implementation effort for both client and server. If this means that 
some RFCs must be corrected to clarify the extension mechanism and make 
it more straightforward,

we should do it.

> Again, my concern here is not to say that the above are ideal
> mechanisms for versioning or similar, but simply to note that the
> existing strict reading is not so fundamentally problematic that it
> necessitates an alternative reading/approach.
[ML]  Maybe it isn't problematic but it seems a bit cumbersome to me.
>> A possible inelegant and misleading workaround would be to treat any new
>> version as a new extension like in the following:
>>
>> {
>>
>> "rdapConformance" [ ....., "ext1", "newfield1"],
>>
>> ...
>> "ext1_data" : {
>>                            ...
>>                           "newfiled1" : { ... }
>>                          },
>> ....
>>
>> }
> If "newfield1" were named e.g. "ext1_1", I'm not sure that it would be
> particularly misleading, at least on a current strict reading of the
> text.
[ML] The example was based on the assumption that there should be a 
tight coupling between the name prefix of a new response extension and 
the rdapConformance value signaling the support for that extension.
> It's probably fair to say that it's inelegant, but I think the
> key thing about this contention between a strict reading and some
> other potential reading is whether the strict reading is so
> problematic that some other reading must have been intended, and I
> don't think this (arguable) inelegance is so serious that that is the
> case here.

[ML] The strict reading results in operational drawbacks that make the 
implementers' work harder.

>
>> Stripping the version information from the extension prefix (Approach B or
>> C)  would simplify the management of this case:
>>
>> {
>>
>> "rdapConformance": [ ....., "ext_1_1"], // (or "ext_level_1_1") to be
>> determined if the RDAP server should include both "ext_1" and "ext_1_1" at
>> least for a period of time
>>
>> ...
>> "ext_data" : {
>>                            ...
>>                           "newfiled" : { ... }
>>                          },
>> ....
>>
>> }
>>
>>
>> Definitively, no breakages in the REST API contract, no deprecation process
>> to be managed by the server, no extra effort for both client and server
>> owing to the fact that JSON is schemeless and, by default, JSON libraries
>> don't block the deserialization of an object including an unknown property.
> The above example would require the extension author to document their
> intent to occupy a namespace of sorts in the extension registry,
> because otherwise a client written to operate against "ext_1" will not
> know what to do when it encounters "ext_1_1" (i.e. it won't be
> possible to make backwards-compatible changes).

[ML] Every change on a stable version implies the change needs to be 
documented and this is should be done regardless the approach defined to 
extend RDAP.

The presence of a minor version in the rdapConformance array would 
signal the clients that a non breaking change have been introduced 
either in the request or in the response.

Client implementers could take all their time to gather information 
about that change and then update their client being confident that, in 
the meantime, it would keep working.

> The idea of an
> extension declaring ownership of a namespace is at odds with the text
> in 9083 about extensions being identified by "a unique string literal
> value registered in the IANA RDAP Extensions registry", too.  Given
> that, I'm not sure that it would be open to take this approach with
> the text as it stands.

[ML] I intended that you, like some others in this WG, agreed that 
current RFCs should be corrected to clarify the argument.

The "conformance" property itself should be defined and such a 
definition should be harmonized with the current RFCs.

If we agree that the RDAP extension mechanism must be reviewed, let's 
define a new one that doesn't introduce ambiguities, drawbacks and is 
the easiest to be implemented.

>> A similar example is about adding a new optional query parameter to an URI
>> path defined by a request extension.
> I think scenario B that I set out previously (i.e. embed further
> versioning information in the extension response) would work for this
> aspect as well.

[ML] Sorry but I didn't catch it. Can you clarify how the presence of a 
new optional query parameter would be signaled in the response?

Anyway, don't see much difference between signaling by adding a new 
version in the rdapConformance array and signaling by introducing an 
ad-hoc response field.



Best,

Mario

> -Tom

-- 
Dr. Mario Loffredo
Technological Unit “Digital Innovation”
Institute of Informatics and Telematics (IIT)
National Research Council (CNR)
via G. Moruzzi 1, I-56124 PISA, Italy
Phone: [phone]
Web: http://www.iit.cnr.it/mario.loffredo