Re: [regext] Extension Prefixes, JSON Values, and URI Path Segments

[Tom Harrison <[e-mail]>]

Hi Mario,

On Fri, May 27, 2022 at 12:41:27PM +0200, Mario Loffredo wrote:
> Think the matter is that even the possible backwards-compatible changes
> would result in being hardly backwards-compatible.
> 
> Let te me give an example to make myself clear and move the discussion on a
> practical perspective.
> 
> Let's call "ext1" the rdapConformance value signaling the support of
> "ext1_data" response extension. The response would be:
> 
> {
> 
> "rdapConformance" : [ ....., "ext1"],
> 
> ...
> "ext1_data" : { ... },
> ....
> 
> }
> 
> Now, let's suppose to add the field "newfield" to "ext1_data" and signal
> this change by a version update from "ext1" to "ext2". The response should
> be (in theory):
> 
> {
> 
> "rdapConformance" [ ....., "ext2"],
> 
> ...
> 
> "ext2_data" : { ... },   // where ext2_data includes all the member of
> "ext1_data" plus "newfield"
> ....
> 
> }
> 
> To avoid a breakage in the REST contract, the RDAP server should implement a
> deprecation process ending with the replacement of "ext1_data" with
> "ext2_data". This means that, for a period of time, both the version of the
> response extensions should be provided:
> 
> {
> 
> "rdapConformance" [ ....., "ext1", "ext2"],
> 
> ...
> "ext1_data" : { ... },
> "ext2_data" : { ... },
> ....
> 
> }
> 
> 
> This situation would result in the following paradox as well: being the
> introduction of new field in a JSON response widely considered a non
> breaking change, it should be signaled by a minor version update (e.g. 
> "ext1_1). But, since the final effect would be the replacement of a response
> field with another,  a major version change should be used instead. In
> short, every new version of the response extension would imply a major
> version update ("ext1", "ext2", "ext3", ..) and  a consequent deprecation
> process the server should support.

I don't understand the relevance of the scenario given above to the
approach I set out for handing backwards-compatible changes.  To avoid
any doubt about the models I was proposing, I'll step through them in
more detail, per your approach above.

 - Scenario A: add single field (backwards-compatible):
    - Initial: register extension "ext1", which notes that "ext1_data"
      will be included in responses, with a single field named
      "field1":

        "rdapConformance": [ ..., "ext1" ],
        ...
        "ext1_data": { "field1": ... }

    - Later: update the existing registration for extension "ext1",
      noting that some implementations of this extension may now
      return "field2".  Implementations in accordance with the
      original version of "ext1" continue working as they do today.
      Implementations in accordance with the new version of "ext1"
      return responses like so:

        "rdapConformance": [ ..., "ext1" ],
        ...
        "ext1_data": { "field1": ...,
                       "field2": ... }

    - The presence of "field2" is enough to distinguish the new
      version from the original version.  Clients written to work with
      the original version do not change, and clients written to work
      with the new version are also fine, because they can determine
      whether the new version has been implemented by way of the
      presence of "field2".

 - Scenario B: add multiple fields (backwards-compatible):
    - Initial: register extension "ext1", which notes that "ext1_data"
      will be included in responses, with a single field named
      "field1".  Subversioning is managed by way of an additional
      "conformance" field, which is used to note minor
      (backwards-compatible) updates:

        "rdapConformance": [ ..., "ext1" ],
        ...
        "ext1_data": { "conformance": "ext1",
                       "field1": ... }

    - Later: update the existing registration for extension "ext1",
      noting that a new "ext1_data.conformance" value of "ext1_1" may
      be used, implementations of which will return "field2" and
      "field3" in the "ext1_data" element.  Implementations in
      accordance with the original version of "ext1" continue working
      as they do today.  Implementations in accordance with the new
      version of "ext1" return responses like so:

        "rdapConformance": [ ..., "ext1" ],
        ...
        "ext1_data": { "conformance": "ext1_1",
                       "field1": ...,
                       "field2": ...,
                       "field3": ... }

    - The "ext1_data.conformance" field is enough to distinguish the
      new version from the original version.  Clients written to work
      with the original version do not change, and clients written to
      work with the new version are also fine, because they can
      inspect at the "conformance" field.

Backwards-incompatible changes continue to require a new top-level
"rdapConformance" value and a transition process where both
"ext1_data" and "ext2_data" (or their equivalents) are returned, per
the example you gave above. 

Again, my concern here is not to say that the above are ideal
mechanisms for versioning or similar, but simply to note that the
existing strict reading is not so fundamentally problematic that it
necessitates an alternative reading/approach.

> A possible inelegant and misleading workaround would be to treat any new
> version as a new extension like in the following:
> 
> {
> 
> "rdapConformance" [ ....., "ext1", "newfield1"],
> 
> ...
> "ext1_data" : {
>                           ...
>                          "newfiled1" : { ... }
>                         },
> ....
> 
> }

If "newfield1" were named e.g. "ext1_1", I'm not sure that it would be
particularly misleading, at least on a current strict reading of the
text.  It's probably fair to say that it's inelegant, but I think the
key thing about this contention between a strict reading and some
other potential reading is whether the strict reading is so
problematic that some other reading must have been intended, and I
don't think this (arguable) inelegance is so serious that that is the
case here.

> Stripping the version information from the extension prefix (Approach B or
> C)  would simplify the management of this case:
> 
> {
> 
> "rdapConformance": [ ....., "ext_1_1"], // (or "ext_level_1_1") to be
> determined if the RDAP server should include both "ext_1" and "ext_1_1" at
> least for a period of time
> 
> ...
> "ext_data" : {
>                           ...
>                          "newfiled" : { ... }
>                         },
> ....
> 
> }
> 
> 
> Definitively, no breakages in the REST API contract, no deprecation process
> to be managed by the server, no extra effort for both client and server
> owing to the fact that JSON is schemeless and, by default, JSON libraries
> don't block the deserialization of an object including an unknown property.

The above example would require the extension author to document their
intent to occupy a namespace of sorts in the extension registry,
because otherwise a client written to operate against "ext_1" will not
know what to do when it encounters "ext_1_1" (i.e. it won't be
possible to make backwards-compatible changes).  The idea of an
extension declaring ownership of a namespace is at odds with the text
in 9083 about extensions being identified by "a unique string literal
value registered in the IANA RDAP Extensions registry", too.  Given
that, I'm not sure that it would be open to take this approach with
the text as it stands.

> A similar example is about adding a new optional query parameter to an URI
> path defined by a request extension.

I think scenario B that I set out previously (i.e. embed further
versioning information in the extension response) would work for this
aspect as well.

-Tom