Re: [regext] Extension Prefixes, JSON Values, and URI Path Segments

[Marc Blanchet <[e-mail]>]

> Le 31 mai 2022 à 15:49, Gould, James <jgould=[e-mail]> a écrit :
> 
> Tom,
>  
> I'm not exactly sure where the term 'strict' model is coming from, which I assume is associated with Approach A "Tight Coupling".  I believe the RFCs are sufficiently unclear to support all three approaches discussed thus far (A, B, and C).  I added “Approach C – decoupled” to the table with commentary with the values for Approach A and B. 


Hello, speaking as RDAP client developer: 
- I’ve been trying to dedicate some time reading this whole thread, but haven’t yet.
- however, just want to say that my client currently does not do anything with these fields, so in the considerations, I would say don’t bother about breaking clients, specially if the new way is a significant improvement.

Regards, Marc.

>  
> Aspect
> Approach A - tight coupling
> Approach B - lack of tight coupling
> Approach  C – decoupled
> Providing a new spec to IANA for a new version
> Registry stays as-is (JG – Only if the prefix does not include versioning)
>  
> JG – Need to update Specification value at a minimum.  If versioning is embedded in the prefix, then it would require new prefix be registered (foo1_2) .  The new prefix Specification value would link to the revised specification.
> Registry needs to evolve for:
> Versioned rdapConformance (JG – I thought that only the prefix was registered and the full version was defined in the specification for the RDAP Conformance)
> Tying spec to versioned rdapConformance (e.g., JG – I believe the registered prefix should always link to the latest version of the specification document, which applies to Approach A and B)
>  
> JG – Need to update Specification value, where the specification document (outside of the RDAP Extension Registry) will define the versioned rdapConformance value (e.g., foo_level_1_2.)
> Registry needs to evolve for:
> Versioned rdapConformance is registered (e.g., foo_level_1_2).
> Prefixes stay as-is unless new prefix registrations are needed by the new specification document.
> Risk of breaking changes for RDAP clients
> Yes, if the server now only supports the new version and a client hasn’t evolved yet
>  
> Data member names and path segments change with each new version but not an issue if clients have re-programmed a priori
>  
> Clients would knowingly call the versioned path segments — no guesswork
>  
> If a path segment is not affected by a new version and only a newly versioned data sub-object is added in the response, that could break clients
>  
> JG – The question is what is the recommended approach for the server to support a new version, should the old and new path segment be support for a period and if the extension is only associated with new response members, should the server support returning both old and new response members for a period?  If the server only supports the new prefix, then it would certainly break the clients that only know about the old version.  The follow-on question is how long the server should support multiple versions?
> Yes, when a field is removed, or a required field is added, for a new version
>  
> JG – I believe it would only be a problem if a field is removed or a required field is made optional.  Adding a new response field (member) should retain backward compatibility.
>  
> When a call is made using a non-versioned path segment, the newly versioned rdapConformance would be checked after the fact and that could break the client for above field additions and subtractions if not re-programmed a priori
>  
> Clients would call non-versioned path segments but could be broken by the new responses
>  
> JG – The server could support more than one version at a time, but that is not guaranteed.  For example, if (foo_level_1 supported the child member “bar” and foo_level_1_2 supported an additional child member “baz”, then the server does support both versions in parallel).  The exact versions supported by the server would provide a hint to the client what is supported in the response.
> Yes, when a field is removed, or a required field is made optional.  If the new version is backward compatible, both the old and new registered rdapConformance identifier can be returned.  
>  
> The path segments and the registered prefixes can remain the same. 
> Cost of reprogramming clients for the next version of an extension
> There is cost but not as high as it seems — replacing version in multiple places and accounting for field and query parameter additions and subtractions
>  
> Longer grace period for clients to reprogram if the server supports multiple versions during transition
>  
> JG – I believe Approach A represents the highest cost to the client with a new version, since all elements of the extension are updated (rdapConformance, path segment, and response members). 
> There is cost — accounting for a field removal or a required field addition
>  
> JG – I believe the cost is associated with making non-backward compatible changes, such as removing a field or making a required field optional.  Removing backward compatibility should be avoided when making a new version of an extension, but that’s not guaranteed.
>  
> Reprogramming could become exigent for clients if the server switches to the new version without supporting the previous version
>  
> JG – Supporting versions in parallel applies to all approaches.  Use of a stable prefix and returning a set of extension version identifiers supported by the extension is less impact to the client than having to change the path segments and the response members for all version changes.
> Yes, when a field is removed, or a required field is made optional.  If the new version is backward compatible, both the old and new registered rdapConformance identifier can be returned.  
>  
> The path segments and the registered prefixes can remain the same, similar to Approach B to reduce the cost over Approach A. 
> Cost of reprogramming servers for the next version of an extension
> There is a higher cost — if multiple versions are simultaneously supported during a transition period, replicating code from the previous version and replacing version in multiple places and accounting for field and query parameter additions and subtractions
>  
> Eventually retiring code for the previous version
> There is a lower cost — if only the latest version is supported, accounting for a field removal or a required field addition vis-a-vis the previous version
> The cost is the same as Approach B, since versioning is not injected into the path segments and response members.
> Server-side signaling of the next version of an extension
> Add the new version of the rdapConformance in the help response and related responses
>  
> Make URLs for new versions of path segments available
> Add the new version of the rdapConformance in the help response and related responses
>  
> No change in URLs for the new version of the rdapConformance
> Same as Approach B.
> Potential confusion for clients
> Zero since the new version is explicitly marked in data members and URLs
>  
> JG – I don’t believe it’s zero, since all that is matched is the prefix.  There is no discovery by the client of the full set of path segments (e.g., foo_segment1, foo_segment2) supported by the server or the meaning of all the response members returned by the server (e.g., foo_data1, foo_data2).  There is concept of auto-discovery built into RDAP, since it’s schemaless, and there is no registry of meaning of the path segment or response member suffixes.  Matching up the prefix in the rdapConformance with the prefix used in the path segments and response members does not provide for auto-discovery by the client.   
> Not zero since the new version is not marked in data members and URLs, and would only be discovered through the rdapConformance value in a response
>  
> JG – Confusion assumes that the client is leveraging some form of auto-discovery, which is not the case.  The rdapConformance value includes only a prefix value and the latest version of the specification document.  The path segments and response members values and meaning is left up to the client to manually extract from the specification document, which is the case for all approaches.
> Same response as for Approach B.
> Aesthetics (does not matter to a machine but could for human friendliness)
> Less
>  
> JG – I believe we’re defining a machine-readable protocol, so I’m not sure whether aesthetics is really a aspect and if so, aesthetics is subjective.  My recommendation is to remove this factor.
> More
>  
> JG – Recommend removal of aspect.
> JG – Recommend removal of aspect.
>  
>  
>  
>  
>  
>  
>  
>  
> Flexibility
> Least
>  
> JG – All path segments and response members use a single prefix value and versioning must be embedded into the single prefix value to support versioning.
> Less
>  
> JG – All path segments and response members use a single prefix value and versioning is restricted to the rdapConformance value, meaning that multiple versions can be supported with a single set of path segments and response members. 
> Most
>  
> The set of rdapConformance identifiers (e.g., foo_level_0, foo_level_1_1, foo_level_2) are independently registered from the prefixes used for the path segments and response members.  More than one prefix can be registered for uniqueness.  The prefixes “foo”, “bar”, and “bar” can be registered for uniqueness using a single RDAP extension specification, and multiple versioned rdapConformance values can be registered. 
>  
>  
>  
>  
>  
>  
> -- 
>  
> JG
>  
>  
>  
> James Gould
> Fellow Engineer
> [e-mail] <mailto:[e-mail]> <applewebdata://13890C55-AAE8-4BF3-A6CE-B4BA42740803/[e-mail] <applewebdata://13890C55-AAE8-4BF3-A6CE-B4BA42740803/[e-mail]>>
>  
> 703-948-3271
> 12061 Bluemont Way
> Reston, VA 20190
>  
> Verisign.com <http://verisign.com/> <http://verisigninc.com/ <http://verisigninc.com/>>
>  
> On 5/30/22, 7:56 AM, "Tom Harrison" <[e-mail] <mailto:[e-mail]>> wrote:
>  
>  
>     Hi Mario,
>  
>     On Mon, May 30, 2022 at 09:51:21AM +0200, Mario Loffredo wrote:
>     > Il 29/05/2022 06:42, Tom Harrison ha scritto:
>     >> On Fri, May 27, 2022 at 12:41:27PM +0200, Mario Loffredo wrote:
>     >>> Think the matter is that even the possible backwards-compatible changes
>     >>> would result in being hardly backwards-compatible.
>     >>> 
>     >>> Let te me give an example to make myself clear and move the discussion on a
>     >>> practical perspective.
>     >>> 
>     >>> Let's call "ext1" the rdapConformance value signaling the support of
>     >>> "ext1_data" response extension. The response would be:
>     >>> 
>     >>> {
>     >>> 
>     >>> "rdapConformance" : [ ....., "ext1"],
>     >>> 
>     >>> ...
>     >>> "ext1_data" : { ... },
>     >>> ....
>     >>> 
>     >>> }
>     >>> 
>     >>> Now, let's suppose to add the field "newfield" to "ext1_data" and signal
>     >>> this change by a version update from "ext1" to "ext2". The response should
>     >>> be (in theory):
>     >>> 
>     >>> {
>     >>> 
>     >>> "rdapConformance" [ ....., "ext2"],
>     >>> 
>     >>> ...
>     >>> 
>     >>> "ext2_data" : { ... },   // where ext2_data includes all the member of
>     >>> "ext1_data" plus "newfield"
>     >>> ....
>     >>> 
>     >>> }
>     >>> 
>     >>> To avoid a breakage in the REST contract, the RDAP server should implement a
>     >>> deprecation process ending with the replacement of "ext1_data" with
>     >>> "ext2_data". This means that, for a period of time, both the version of the
>     >>> response extensions should be provided:
>     >>> 
>     >>> {
>     >>> 
>     >>> "rdapConformance" [ ....., "ext1", "ext2"],
>     >>> 
>     >>> ...
>     >>> "ext1_data" : { ... },
>     >>> "ext2_data" : { ... },
>     >>> ....
>     >>> 
>     >>> }
>     >>> 
>     >>> 
>     >>> This situation would result in the following paradox as well: being the
>     >>> introduction of new field in a JSON response widely considered a non
>     >>> breaking change, it should be signaled by a minor version update (e.g.
>     >>> "ext1_1). But, since the final effect would be the replacement of a response
>     >>> field with another,  a major version change should be used instead. In
>     >>> short, every new version of the response extension would imply a major
>     >>> version update ("ext1", "ext2", "ext3", ..) and  a consequent deprecation
>     >>> process the server should support.
>     >> 
>     >> I don't understand the relevance of the scenario given above to the
>     >> approach I set out for handing backwards-compatible changes.  To avoid
>     >> any doubt about the models I was proposing, I'll step through them in
>     >> more detail, per your approach above.
>     >> 
>     >>   - Scenario A: add single field (backwards-compatible):
>     >>      - Initial: register extension "ext1", which notes that "ext1_data"
>     >>        will be included in responses, with a single field named
>     >>        "field1":
>     >> 
>     >>          "rdapConformance": [ ..., "ext1" ],
>     >>          ...
>     >>          "ext1_data": { "field1": ... }
>     >> 
>     >>      - Later: update the existing registration for extension "ext1",
>     >>        noting that some implementations of this extension may now
>     >>        return "field2".  Implementations in accordance with the
>     >>        original version of "ext1" continue working as they do today.
>     >>        Implementations in accordance with the new version of "ext1"
>     >>        return responses like so:
>     >> 
>     >>          "rdapConformance": [ ..., "ext1" ],
>     >>          ...
>     >>          "ext1_data": { "field1": ...,
>     >>                         "field2": ... }
>     >> 
>     >>      - The presence of "field2" is enough to distinguish the new
>     >>        version from the original version.  Clients written to work with
>     >>        the original version do not change, and clients written to work
>     >>        with the new version are also fine, because they can determine
>     >>        whether the new version has been implemented by way of the
>     >>        presence of "field2".
>     > 
>     > [ML] Usually, REST-JSON consumers configure JSON libraries to ignore unknown
>     > fields in deserialization. This just to reduce to prevent from breaking
>     > changes in responses.
>     > 
>     > Hence, the presence of "field2" wouldn't be detected unless it was managed
>     > by the client (e.g. by putting every unknown field on a given map).
>  
>     I'm not sure how this is different from what I wrote.  A client
>     configured in this way is one that works against the original version
>     only, which is fine.  If/when the client is updated to support the new
>     version, then it can make use of the new field.
>  
>     >>   - Scenario B: add multiple fields (backwards-compatible):
>     >>      - Initial: register extension "ext1", which notes that "ext1_data"
>     >>        will be included in responses, with a single field named
>     >>        "field1".  Subversioning is managed by way of an additional
>     >>        "conformance" field, which is used to note minor
>     >>        (backwards-compatible) updates:
>     >> 
>     >>          "rdapConformance": [ ..., "ext1" ],
>     >>          ...
>     >>          "ext1_data": { "conformance": "ext1",
>     >>                         "field1": ... }
>     >> 
>     >>      - Later: update the existing registration for extension "ext1",
>     >>        noting that a new "ext1_data.conformance" value of "ext1_1" may
>     >>        be used, implementations of which will return "field2" and
>     >>        "field3" in the "ext1_data" element.  Implementations in
>     >>        accordance with the original version of "ext1" continue working
>     >>        as they do today.  Implementations in accordance with the new
>     >>        version of "ext1" return responses like so:
>     >> 
>     >>          "rdapConformance": [ ..., "ext1" ],
>     >>          ...
>     >>          "ext1_data": { "conformance": "ext1_1",
>     >>                         "field1": ...,
>     >>                         "field2": ...,
>     >>                         "field3": ... }
>     >> 
>     >>      - The "ext1_data.conformance" field is enough to distinguish the
>     >>        new version from the original version.  Clients written to work
>     >>        with the original version do not change, and clients written to
>     >>        work with the new version are also fine, because they can
>     >>        inspect at the "conformance" field.
>     > 
>     > [ML]  As a  consequence of this scenario,  every RDAP response extension
>     > should include a fixed property named "conformance".
>  
>     Not necessarily.  An extension author might decide that the chance of
>     change is so low that they don't see the need to incorporate this sort
>     of signalling into their specification.
>  
>     > If this could be admissible for response extensions defined in the RDAP
>     > context, it couldn't fit for external specifications.
>     > 
>     > See JSContact for example. VCard itself is an external specification used
>     > for defining an RDAP response.
>  
>     Wrapping the external specification object inside another object that
>     provides the version/conformance information for that wrapped object
>     seems like an adequate way to deal with this concern.
>  
>     >> Backwards-incompatible changes continue to require a new top-level
>     >> "rdapConformance" value and a transition process where both
>     >> "ext1_data" and "ext2_data" (or their equivalents) are returned, per
>     >> the example you gave above.
>     > 
>     > [ML]  Both Scenario A and B seem to me requiring some additional effort from
>     > implementers. Can't understand why we should make more difficult what could
>     > be managed straightforwardly.
>     > 
>     > Think we should accomplish the solution requiring the lowest (to zero)
>     > implementation effort for both client and server. If this means that some
>     > RFCs must be corrected to clarify the extension mechanism and make it more
>     > straightforward, we should do it.
>  
>     There are a few considerations here:
>  
>      - There appears to be general consensus that clarification of the
>        extension mechanism in RDAP is required.  At one end, that
>        clarification could restate the 'strict' model more clearly.  At
>        the other, it might be something like James' "Approach C", from
>        https://secure-web.cisco.com/1ulhGgq9JP8ffpf10HZPcsFR6a2t_1y2oqCnQDCHrmHfAwxuYA2CkNC8wIoJr1wMmzBlChqvsYLaQiok08gIVgVmnmhOWqMpBgGB_zDpkQEuIscd6mZHDaX01RhgmZtrljMj1S4vipjCpYUI6wCWmoFSFbaFd5cPodoNXMtN40X7ckY0VyLCu1sYmP5II5kq_qLWsbgt_0mdLGpSgegXllJyMHgjrbM31LzJ1dYgt2tGPP4JzWgtsauNHec-KmFl7/https%3A%2F%2Fmailarchive.ietf.org%2Farch%2Fmsg%2Fregext%2Fe7hZPYUoD3GmjdGiC444TmrCsjg%2F <https://secure-web.cisco.com/1ulhGgq9JP8ffpf10HZPcsFR6a2t_1y2oqCnQDCHrmHfAwxuYA2CkNC8wIoJr1wMmzBlChqvsYLaQiok08gIVgVmnmhOWqMpBgGB_zDpkQEuIscd6mZHDaX01RhgmZtrljMj1S4vipjCpYUI6wCWmoFSFbaFd5cPodoNXMtN40X7ckY0VyLCu1sYmP5II5kq_qLWsbgt_0mdLGpSgegXllJyMHgjrbM31LzJ1dYgt2tGPP4JzWgtsauNHec-KmFl7/https%3A%2F%2Fmailarchive.ietf.org%2Farch%2Fmsg%2Fregext%2Fe7hZPYUoD3GmjdGiC444TmrCsjg%2F>
>      - The closer that clarification is to the current 'strict' model, the
>        better (IMHO), since that will help to minimise change in existing
>        RDAP clients.
>      - If people are noting apparent limitations of the 'strict' model,
>        then demonstrating that the limitations may not be so serious as
>        initially thought may help with keeping something like the 'strict'
>        model in place.
>  
>     Of course, the consensus may be that more extensive change is
>     worthwhile notwithstanding that the 'strict' model can be made to
>     accommodate the various requirements that people are raising here.
>     However, I think it's useful all the same to talk about what can and
>     can't be done with that model today.
>  
>     (The above is also relevant to the question of how the existing
>     documents should be interpreted, separately from a potential
>     clarifying document.  For example, if the 'strict' model meant that
>     backwards-compatible changes were as onerous as you posited, that
>     would be an argument in favour of an alternative reading.)
>  
>     >> Again, my concern here is not to say that the above are ideal
>     >> mechanisms for versioning or similar, but simply to note that the
>     >> existing strict reading is not so fundamentally problematic that it
>     >> necessitates an alternative reading/approach.
>     > 
>     > [ML]  Maybe it isn't problematic but it seems a bit cumbersome to me.
>     > 
>     >>> A possible inelegant and misleading workaround would be to treat any new
>     >>> version as a new extension like in the following:
>     >>> 
>     >>> {
>     >>> 
>     >>> "rdapConformance" [ ....., "ext1", "newfield1"],
>     >>> 
>     >>> ...
>     >>> "ext1_data" : {
>     >>>                            ...
>     >>>                           "newfiled1" : { ... }
>     >>>                          },
>     >>> ....
>     >>> 
>     >>> }
>     >> 
>     >> If "newfield1" were named e.g. "ext1_1", I'm not sure that it would be
>     >> particularly misleading, at least on a current strict reading of the
>     >> text.
>     > 
>     > [ML] The example was based on the assumption that there should be a tight
>     > coupling between the name prefix of a new response extension and the
>     > rdapConformance value signaling the support for that extension.
>  
>     Fair enough.  Scenario A is still an adequate way of dealing with the
>     concern here, I think.
>  
>     >> It's probably fair to say that it's inelegant, but I think the
>     >> key thing about this contention between a strict reading and some
>     >> other potential reading is whether the strict reading is so
>     >> problematic that some other reading must have been intended, and I
>     >> don't think this (arguable) inelegance is so serious that that is the
>     >> case here.
>     > 
>     > [ML] The strict reading results in operational drawbacks that make the
>     > implementers' work harder.
>     > 
>     >>> Stripping the version information from the extension prefix (Approach B or
>     >>> C)  would simplify the management of this case:
>     >>> 
>     >>> {
>     >>> 
>     >>> "rdapConformance": [ ....., "ext_1_1"], // (or "ext_level_1_1") to be
>     >>> determined if the RDAP server should include both "ext_1" and "ext_1_1" at
>     >>> least for a period of time
>     >>> 
>     >>> ...
>     >>> "ext_data" : {
>     >>>                            ...
>     >>>                           "newfiled" : { ... }
>     >>>                          },
>     >>> ....
>     >>> 
>     >>> }
>     >>> 
>     >>> 
>     >>> Definitively, no breakages in the REST API contract, no deprecation process
>     >>> to be managed by the server, no extra effort for both client and server
>     >>> owing to the fact that JSON is schemeless and, by default, JSON libraries
>     >>> don't block the deserialization of an object including an unknown property.
>     >> 
>     >> The above example would require the extension author to document their
>     >> intent to occupy a namespace of sorts in the extension registry,
>     >> because otherwise a client written to operate against "ext_1" will not
>     >> know what to do when it encounters "ext_1_1" (i.e. it won't be
>     >> possible to make backwards-compatible changes).
>     > 
>     > [ML] Every change on a stable version implies the change needs to be
>     > documented and this is should be done regardless the approach defined to
>     > extend RDAP.
>     > 
>     > The presence of a minor version in the rdapConformance array would signal
>     > the clients that a non breaking change have been introduced either in the
>     > request or in the response.
>     > 
>     > Client implementers could take all their time to gather information about
>     > that change and then update their client being confident that, in the
>     > meantime, it would keep working.
>     > 
>     >> The idea of an
>     >> extension declaring ownership of a namespace is at odds with the text
>     >> in 9083 about extensions being identified by "a unique string literal
>     >> value registered in the IANA RDAP Extensions registry", too.  Given
>     >> that, I'm not sure that it would be open to take this approach with
>     >> the text as it stands.
>     > 
>     > [ML] I intended that you, like some others in this WG, agreed that current
>     > RFCs should be corrected to clarify the argument.
>     > 
>     > The "conformance" property itself should be defined and such a definition
>     > should be harmonized with the current RFCs.
>     > 
>     > If we agree that the RDAP extension mechanism must be reviewed, let's define
>     > a new one that doesn't introduce ambiguities, drawbacks and is the easiest
>     > to be implemented.
>  
>     Per your comments, I was concerned to note that this proposed approach
>     would not work as against the current text, rather than that it was a
>     bad idea in the context of a broader rethinking of extensions in RDAP.
>  
>     >>> A similar example is about adding a new optional query parameter to an URI
>     >>> path defined by a request extension.
>     >> 
>     >> I think scenario B that I set out previously (i.e. embed further
>     >> versioning information in the extension response) would work for this
>     >> aspect as well.
>     > 
>     > [ML] Sorry but I didn't catch it. Can you clarify how the presence of a new
>     > optional query parameter would be signaled in the response?
>  
>     To work through another scenario:
>  
>      - Initial: register extension "ext1", which notes that the "help"
>        response will include an additional "ext1_version" element
>        describing the exact version of the specification being used
>        (initially "1.0").  The extension also supports a new path,
>        "/ext1_query".
>      - Later: update the existing registration for extension "ext1",
>        noting that "ext1_version" is now "2.0", and that the "/ext1_query"
>        path now supports the new optional parameter.
>      - Clients written to work with the original version do not change.
>        Clients written to work with the new version can inspect the
>        "ext1_version" field in order to determine whether the server
>        supports the optional parameter.
>  
>     > Anyway, don't see much difference between signaling by adding a new
>     > version in the rdapConformance array and signaling by introducing an
>     > ad-hoc response field.
>  
>     The difference is that the ad hoc response field can be used with the
>     current 'strict' model.
>  
>     -Tom
>  
> _______________________________________________
> regext mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/regext <https://www.ietf.org/mailman/listinfo/regext>