[regext] Re: Fwd: New Version Notification for draft-newton-regext-rdap-considerations-on-rfc9537-00.txt

["Gould, James" <[e-mail]>]

Andy,

As noted in my prior response, below is my detailed responses to draft-newton-regext-rdap-considerations-on-rfc9537:


  1.  Abstract
     *   “Some of these problems may be insurmountable, leaving portions of RFC 9537 non-interoperable between clients and servers, while other problems place a high degree of complexity upon clients.”

                                                              i.      The RFC provides a set of required members for each redaction with the “name” and “method” members, along with a set of optional members.  Your considerations focus on the implementation of the optional members that use an expression language (JSONPath) to technically identify the JSON members redacted via the supported methods.  We’ve had a lot of discussion on the list with the inherent complexity of RDAP and jCard containing structured and unstructured content that any JSON expression language will have challenges with.  I still believe that JSONPath is the appropriate expression language to use currently, but the RFC supports extensibility of expression languages if a better expression language is created.  At a minimum, the client should be capable of easily leveraging the required “name” and “method” members to provide a list of redactions to the end users with optional visual field-level indication, which is not insurmountable and can be done in an interoperable way.

  1.  Background
     *   “This RFC also modifies the IANA RDAP JSON Values registry with additional fields to be used in the process of presenting information about a redaction, however the registry contains text intended for humans and does not explicitly contain JSONPath expressions.”

                                                              i.      Correct, the RFC leverages the existing RDAP JSON Values IANA registry with the new types of “redacted name”, “redacted reason”, and “redacted expression language”.  The registry is meant for humans and not programmatic discovery, which meets the needs for these new types.  In particular, the “redacted name” entries will provide the standard names that clients can key off.  The latest “redacted name” entries include the use a JSONPath in the Description field that can be applied against a static response in the RFC for clarity purposes only.

     *   “clients merely rendering the values of "prePath", "postPath", or "replacementPath" are of little use to most humans.”

                                                              i.      Agreed, the “name” values and especially the standard “name” values, using the IANA RDAP JSON Values registry, is the most appropriate to display to the end user.  The “name” values can be used by the client to highlight a field displayed to the user that has been redacted, such as including the “Registry Domain ID” field without a value and with red strikeout text if the “Registry Domain ID” registered “name” value is included in the redaction extension with a “method” value of “removal”.

  1.  Redaction by Removal
     *   “Further, [RFC9537<https://www.rfc-editor.org/info/rfc9537>] makes "prePath" OPTIONAL and does not require its usage, though every example in the RFC of redaction by removal does use "prePath".”

                                                              i.      The “path “member, which was expanded, was required in an early draft, but was made optional in place of the required “name” member.  Inclusion of the path members in the examples, should not provide any indication to the implementer that an OPTIONAL member is a SHOULD or a MUST.

     *   “In other words, the client may be able to know the information that was removed using the “name” member of the redaction directive described in Section 4.2 of [RFC9537<https://www.rfc-editor.org/info/rfc9537>] though the RFC does not explain how the client is to go about doing this nor does it use normative RFC language to indicate this is the nature of the mechanism to use.”

                                                              i.      The RFC defines the “name” as the logical name and supports the registration of the “name” values that provides the Description in the IANA RDAP JSON Values.  The unregistered “name” values are up to server policy, which means that the server will need to communicate the meaning of the unregistered “name” values.  Extra clarify could have been provided for the unregistered “name” values by recommending that the server document and provide the list of “name” values for use by clients.  The registered “redacted name” values should provide the needed clarify for client implementations in the Description field.

     *   “The “type” string contains a value registered with IANA in the RDAP JSON Values registry. This registry does not contain any formalism to describe the data to be redacted. The “description” string contains any text and has no definition. Therefore, RFC 9537 does not describe a mechanism for a client implementer to "explicitly" determine the information that is to be redacted.”

                                                              i.      The registered “redacted name” values look to provide a clear description of what is redacted by including a human readable description along with the use of a JSONPath expression against a static RDAP response in the RFC.  It’s not up to the RFC to define what will be a useful description in the RDAP JSON Values registry, but it’s up to those registering them and the Designated Experts reviewing them.  The “description” string is up to server policy and as noted in 3.b above, the server can document and make available to clients.  My recommendation is for the servers to registry the “redacted name” values so that the clients can build support for them explicitly.

     *   “Client implementers must therefore devise a method not given by RFC 9537.”

                                                              i.      “One possible solution is to have the client dynamically generate a JSONPath expression for each value it would normally present and then compare that value against the “prePath” string.”

           *   This is not a recommended option based on the level of complexity.

                                                             ii.      “Another possible solution would be to have the client dynamically construct an RDAP JSON response based on the “prePath” string.”

           *   I don’t see any mechanism for the client to dynamically construct an RDAP JSON from the JSONPath expressions.
           *   I believe the only reasonable option to display the unredacted response with the actual data of the response and with a visual indication of the redaction, is to use a template unredacted response from the server.  There is no standard of the exact RDAP response provided by all servers, so this would need to be done on a per-server basis.  We included in -10 the JSONPath Client Consideration language “The client can first key off the "name" member for display logic and utilize a template RDAP response overlaid with the redacted response to successfully resolve the JSONPath expression.”, which was removed in -12 based on feedback from Tom Harrison.  Maybe we should bring back a modified form of this consideration.

                                                           iii.      “Therefore, for all practical purposes a client has no known method to "explicitly" identify the values of an RDAP response that have been redacted.”

           *   Please explain why the clients can’t leverage the required “name” and “method” members to provide a list of redactions to end users or why it’s technically infeasible to provide a visual indication with the list of fields provided to the end users?  The end users in the ICANN Lookup Tool get presented with a WHOIS-like visualization, so it really doesn’t need display the JSON with and without the redaction.

                                                           iv.      “Finally, the implication of the "prePath" expression is that it should always evaluate to an empty set.”

           *   No, the “prePath” needs to be applied to the unredacted response, which would evaluate to a non-empty set.
  1.  Redaction by Replacement Value
     *   “Nor does RFC 9537 provide guidance to client implementers as to the purpose of signaling the replacement (i.e. what are clients to do with this information).”

                                                              i.      The purpose is strictly to provide a signal to the client that information was changed by the server.  It’s up to client policy what to do with this information.

     *   “There is no guidance given to client implementers as to the purpose of this information. That is, the information is present in the response but there is no formal definition of what has been replaced.”

                                                              i.      If there is no “prePath”, the member value itself has been changed / replaced by the server.  The server’s policy was to replace the information, which would be hidden to the client without the redaction extension.  The redaction extension will inform the client that the value has been changed and there is no need for the server to disclose what the unredacted value was.  It’s up to client policy what to do with this information.

     *   “However, the structure of the redaction JSON does not allow for multiple descriptions with differing “lang” tags and therefore this mechanism has some internationalization problems.”

                                                              i.      The “reason” member includes support for a “lang” member with a default language “en”, which would support providing a different language if the RDAP language could specify their desired language.  Do you know a mechanism for the RDAP client to specify their desired language?  Is there an example of where in the RDAP RFCs or the RDAP extensions with supporting the return of more than one language at a time?

  1.  Types and Empty Value
     *   “This text does not fully describe the nature of an empty value for all JSON types, instead equating an empty string as an empty value for JSON strings and null as the empty value for all other JSON types.”

                                                              i.      The working group specifically worked out this language based on support of jCard.  It states, “The Redaction by Empty Value Method MUST be used only when redacting JSON response fields that use the position in an array to signal the redacted field (e.g., jCard arrays).”  The broader examples that you provide in the draft MUST NOT use the Empty Value Method.

     *   “In addition to this, null is not the same type as other JSON types and therefore cannot be used anywhere a specific type, such as a boolean or number, has been specified unless explicitly allowed by the JSON-using specification. That is, if RFC 9083 designates a value as a specific type a server cannot change that value to a null in an RDAP response and be compliant with [RFC9083<https://www.rfc-editor.org/info/rfc9083>].“

                                                              i.      null (empty) is a valid JSON data type that goes along with string, number, object, array, and boolean.  I view null (empty) as a valid alternative to number and boolean in JSON.  Please provide an RFC and RFC language that disallows the use of a null value for a boolean or number field in JSON.  I don’t view a server providing a null value for a number or boolean jCard field as being non-compliant with JSON or RFC 9083.

     *   “Within the core RDAP responses, null is not allowed in any place. Therefore, using it would create RDAP invalid JSON, which is also not allowed by this RFC as stated in Section 3:”

                                                              i.      Where in the core RDAP RFCs does it disallow the use of a JSON null value?

                                                             ii.      I found in RFC 9083 the following:

           *   “simple data types conveyed in JSON primitive types (strings, numbers, booleans, and null)”
           *   “JSON [RFC8259<https://datatracker.ietf.org/doc/html/rfc8259>] defines the data types of a number, character string, boolean, array, object, and null.”
           *   “The "fn" member is required and MUST NOT be null according to [RFC6350<https://datatracker.ietf.org/doc/html/rfc6350>] An empty "fn" member MAY be used when the contact name does not exist or is redacted.”
     *   For all practical purposes, redaction by empty value can only applied to a JSON string.

                                                              i.      This is false, based on 5.c above.

  1.  Types and Partial Values
     *   “the RFC does not describe the semantics of partial value redaction for any but the string type. Therefore, clients are given no guidance regarding the nature of a partial boolean, number, or null.

                                                              i.      The RFC includes the signal that a value is not provided completely.  There is no mechanism to describe what part of the value was redacted.  The client can indicate that the field was modified (not replaced) by the server due to redaction.  This is a complexity left over by jCard that the redaction extension addresses with the Partial Value Method

     *   “Furthermore, when applying this redaction method to a string, object, or array a client has no way of determining which parts were redacted. The example given in the RFC is of the string “Vancouver\nBC\n1239\n" but the client has no means of determining if the redacted portion of the string comes before, after, or in the middle of this string.

                                                              i.      The Partial Value was added to cover the corner case of jCard unstructured fields.  We could look to adjust the language in the RFC to have the Partial Value Method only apply to the unstructured jCard fields to provide a coarse-grained indication of redaction.  I don’t recommend attempting to use another expression language, such as RegEx to indicate which part of the unstructured field was redacted, which again would need to be applied to the unredacted template value.

     *   Therefore, for all practical purposes there is no distinction between redaction by partial value and redaction by empty value. Furthermore, if a client cannot determine which parts of a string have been redacted, then it cannot adequately present this information to a user.

                                                              i.      The client can provide the redaction list or provide an indication of redaction by type at the field level, like the fields (e.g., “Registry Domain ID”) presented in the ICANN Lookup Tool.

  1.  Overlapping and Nesting Redactions
     *   Another issue that is unaddressed by RFC 9537 is that of overlapping JSONPath expressions.

                                                              i.      The RFC provides the tools for the server to include the list of redactions, but it’s up to the implementers to make the right decisions.  RFC 9083 doesn’t address duplicate or inconsistent information being provided by the server.   The example provided is obviously an implementation bug of the server that needs to be fixed, but the RFC doesn’t need to cover all implementation cases that may come up.

  1.  JSONPath Implementation Conformation
     *   Thanks for doing the research into the JSONPath libraries.  JSONPath recently became an RFC, so hopefully the libraries will catch up.
  2.  Tel URIs
     *   “Should a server redact the phone extension, how does the tel URI get expressed: tel:[phone];ext= or tel:[phone]?”

                                                              i.      If the phone extension is redacted, my recommendation is to return tel:[phone]” with the redaction name “Registrant Phone Ext” and redaction method “partialValue”.   The client can include that in the list of redactions and provide a visual indication at the field level.

     *   “the "tel" property is not required to be a tel URI but maybe unstructured text.”

                                                              i.      This is the complexity of jCard that we’re dealing with in general with RDAP.  For redaction, if the voice extension is redacted using the unstructured “tel” jCard field, the same redaction name “Registrant Phone Ext” and redaction method “partialValue” can be used for display to the end user.

     *   “As the string to be redacted by partial value may take many forms, clients have no formal means of determine which part is to be redacted.”

                                                              i.      The client can determine what part was redacted if the server returned back the redaction name “Registrant Phone Ext” to indicate that the extension was redacted.  If the client displays the voice number together with the extension, then a partial makes sense, but if the client displays the voice number separately from the voice extension, then it could be changed to a “removal” to the end user by the client.

  1.  Structured and Unstructured Addresses
     *   “Additionally, if the server references an IANA registered redaction, the registration should cover both cases.”

                                                              i.      The JSONPath included in the “redacted name” registration is not meant to be authoritative to cover all RDAP response, but simply to provide a description for clients and servers on the meaning of the registered “redacted name” value.  The key for the registrations is to be easy to understand their meaning with the recommendation to provide a reference to a concrete unredacted RDAP response for clarity.

  1.  Complexity
     *   “Given this, it is unlikely that any general purpose RDAP client would be implementing [RFC9537<https://www.rfc-editor.org/info/rfc9537>] without explicit, directed funding for that purpose. And it is likely that implementation among clients will differ substantially in behavior.”

                                                              i.      What you mean is a generate purpose RDAP client that displays the RDAP response graphically with and without redaction.  I believe clients like the ICANN Lookup Tool, transforms the RDAP response into a readable form for the end user.  The ICANN Lookup Tool also provides the raw RDAP responses, which should remain unmodified.  All that is needed is to provide a list of the redactions that is transformed in a similar fashion, with the option of providing visual indication of the redactions in the fields themselves.  Providing the visual indication of the redacted fields can be done in many ways.

  1.  Path Forward
     *   “To summarize the findings:”

                                                              i.      “Redaction by removal cannot be accomplished by "explicitly identifying" the parts of an RDAP JSON response that have been removed.”

           *   Redaction by removal with the use of “prePath” is optional and can be used if the client has access to a template unredacted response.  The redaction by removal can be used to provide the redaction done to the end users explicitly in a list.

                                                             ii.      “Redaction by empty value is only applicable to JSON strings.”

           *   I disagree with this statement.  Please provide the RFC language that disallows the use of a null value of number and boolean JSON types.

                                                           iii.      “Redaction by partial value cannot specify the parts of the value redacted, is only applicable to strings, and therefore no different than redaction by empty value.”

           *   The redaction by partial value is different from redaction by empty value since the entire value has not been removed but has been modified.  For the ICANN Lookup Tool, fields that were redacted partially can be listed and visually indicated at the field level.

                                                           iv.      “Redaction by replacement value cannot identify the values that were replaced.”

           *   In one of the cases, it matches the redaction by removal feedback, where the client would need a template unredacted response to provide a visual indication in JSON.  At a minimum, the client can leverage the redaction by replacement to provide the method of redaction done to the end user.

                                                             v.      “The usage of JSONPath is too broad, rendering the mechanisms of [RFC9537<https://www.rfc-editor.org/info/rfc9537>] that do work to be non-interoperable.”

           *   This is too broad of a statement.  The draft includes many examples of where JSONPath can be used.  In the short term, for your clients my recommendation is to not leverage JSONPath, since it is not meeting your needs or the needs of your users.  I actually use the ICANN Lookup Tool all the time, and I would see value in simply providing the list of redactions and to not touch the raw RDAP responses from the registry and the registrar.

                                                           vi.      “Overlaps in JSONPath expressions can present challenges to clients attempting to display textual explanations.”

           *   This is a bug in the server implementation that needs to be addressed by the server.
     *   “One potential fix would be to instruct clients to always ignore the JSONPath expressions and instead strictly key off the redacted registrations in the IANA registry. The implications of this approach is that each registration would need to clearly specify the parts of RDAP to be redacted, and client implementers would be required to update their client software as frequently as necessary to accommodate new registrations. In other words, this approach would not "explicitly identify" redacted RDAP data, nor would it give servers the ability to express redaction reasons in multiple languages.”

                                                              i.      The JSONPath is optional and can be used if the server provides template unredacted response for the client to use, which is consistent with the format used in the redacted response.  In the case of the ICANN clients and the gTLD registries and registrars, my recommendation is to not use the JSONPath members (“prePath”, “postPath”, “replacementPath”, and “pathLang”) and to leverage the “name” with registered “redacted name” values and “method” methods for a much better redaction signal than the existing “REDACTED FOR PRIVACY” remark.

                                                             ii.      The clients can count on the registered “redacted name” values and can also display the unregistered “name” values that use the “description” field to provide the end user with the additional redaction information.  My recommendation is to only provide visual field indication (e.g., red strikeout) for the registered “redacted name” values, since they should be well known and can be mapped to the display fields at the time of development.  I don’t foresee the RDAP JSON Values registry getting updated frequently, so it should be manageable.   For example, the ICANN Lookup Tool can key off the redaction of the “Registry Domain ID” to include in the list of redactions to the end user, along with red strikeout text of the “Registry Domain ID” field.  Unregistered names using the “description” field or unknown registered names using the “type” field can be included in the list of redactions without the additional field-level visual indication.  This approach does enable the server to indicate what has been redacted using registered and unregistered “name” values, with a predefined set of redaction methods, and with the option of providing a redaction reason is the server’s defined language.
“--

JG

[cid87442*image001.png@01D960C5.C631DA40]

James Gould
Fellow Engineer
[e-mail]<applewebdata://13890C55-AAE8-4BF3-A6CE-B4BA42740803/[e-mail]>

703-948-3271
12061 Bluemont Way
Reston, VA 20190

Verisign.com<http://verisigninc.com/>

From: James Gould <[e-mail]>
Date: Wednesday, May 29, 2024 at 8:57 AM
To: "[e-mail]" <[e-mail]>, "[e-mail]" <[e-mail]>
Subject: Re: [regext] Fwd: New Version Notification for draft-newton-regext-rdap-considerations-on-rfc9537-00.txt

Andy,


Thank you for providing information on the client implementation experience.  I will review all the content in detail to provide my thoughts on how to address the issues.  Much of your feedback is associated with the complexities of processing the JSONPath expressions, which provides an optional hint to the clients.  The RDAP JSON responses are inherently complex with the mix of structured and unstructured content with jCard.



At a minimum, the client can re-display the list of redactions to the end user in a user-friendly manner by keying off the “name” and “method” members.  A standard set of “name” values can be defined by registering the “redacted name” RDAP JSON Values, which include a description that is meant for manual review by client implementors and not programmatic discovery.  Such as the following for Figure 12: “Redacted RDAP Lookup Response” in RFC 9537:



Redacted by Removal:



  *   Registry Domain ID
  *   Registrant Name
  *   Registrant Organization
  *   Registrant Street
  *   Registrant City
  *   Registrant Postal Code
  *   Registrant Email
  *   Registrant Phone
  *   Technical Name
  *   Technical Email
  *   Technical Phone
  *   Technical Fax
  *   Administrative Contact
  *   Billing Contact



The “emptyValue” and “removal” method are different forms of removal, so they can be grouped in the removal bucket for the end users.



Is there any other experience from client implementers (e.g., Marc Blanchet) that can be shared?

Thanks,

--

JG

[cid87442*image001.png@01D960C5.C631DA40]

James Gould
Fellow Engineer
[e-mail]<applewebdata://13890C55-AAE8-4BF3-A6CE-B4BA42740803/[e-mail]>

703-948-3271
12061 Bluemont Way
Reston, VA 20190

Verisign.com<http://verisigninc.com/>

From: "Andrew Newton (andy)" <[e-mail]>
Date: Wednesday, May 29, 2024 at 6:51 AM
To: "[e-mail]" <[e-mail]>
Subject: [EXTERNAL] [regext] Fwd: New Version Notification for draft-newton-regext-rdap-considerations-on-rfc9537-00.txt


Hi all,

Over the past several months, we have been implementing the RDAP redaction extension, RFC 9537.

This I-D describes the issues we have encountered.

-andy


-------- Forwarded Message --------
Subject:

New Version Notification for draft-newton-regext-rdap-considerations-on-rfc9537-00.txt

Date:

Wed, 29 May 2024 03:45:43 -0700

From:

[e-mail]<mailto:[e-mail]>

To:

Andy Newton <[e-mail]><mailto:[e-mail]>



A new version of Internet-Draft
draft-newton-regext-rdap-considerations-on-rfc9537-00.txt has been
successfully submitted by Andy Newton and posted to the
IETF repository.

Name: draft-newton-regext-rdap-considerations-on-rfc9537
Revision: 00
Title: Considerations on RFC 9537
Date: 2024-05-29
Group: Individual Submission
Pages: 12
URL: https://www.ietf.org/archive/id/draft-newton-regext-rdap-considerations-on-rfc9537-00.txt<https://secure-web.cisco.com/1mPMfNI2zob5OY_RHSNl_P1tnUeETO0o09c0W99TATan4s3PHChO6Kh1GiGUrfpIOmq6rKMwYi0JSWGkAQs2-cjvAkdc7uWAKWTZ_6BxEYYPPJgxWQILWPHRcZ_Tpd5HTVE30WZyezOXpEI0mSmbMrUK_Q30sXwoF87IHofTbGV7STLZFRkmmNTofEO74X4EQY6enivux3oILbYyXDii0CLePuRqOAZ4Qp3aWC4SLOnsSriEi-34rlQSYQIjI6h7_Q-SH5FjXk1QGLcdhrptCiGgVCA6f7sRzI_WC4NHhZtk/https%3A%2F%2Fwww.ietf.org%2Farchive%2Fid%2Fdraft-newton-regext-rdap-considerations-on-rfc9537-00.txt>
Status: https://datatracker.ietf.org/doc/draft-newton-regext-rdap-considerations-on-rfc9537/<https://secure-web.cisco.com/1fNr0Hl9vsQaWidCOrR4dk8_7jjg-1TO_r1Bszl7rGuy8SKgw9GD6cCHI2YQnFxZdFFx5WZupc9gOjmLjjgcXrwyw-FTAWtkrPceSmR8xS87wUYeveCOq1QLrIj4GDwrSRyf8sAFwiQYIsMI6NTjxHl8mRM92bJEyMP7j64D3uMXkuv_wKVyOaHvmd7-C272ah5DqW4YenHArcCZ5tON4CO43CGN2Gra2u9d5V6C6ARisZgLcHvKN4LtnkGoIqHmLHj0ZoH-8EY_r6zhnNgmb2DIOfwC6iYe8JRpJDQSsjx4/https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-newton-regext-rdap-considerations-on-rfc9537%2F>
HTML: https://www.ietf.org/archive/id/draft-newton-regext-rdap-considerations-on-rfc9537-00.html<https://secure-web.cisco.com/1aste1pdAhq_TMFqsj5FriZl0Uu9f059LMSBJJnZqQV8xMWJnw9mkTR_sut4SqV4A8VXdSyu1ww4Z2J2gjs5GBNL61WGWNc_vpADz1cHfghH-SEa5EVH9VBgPFz8_Ew3N05VZTPteEm5fSDNjw_XFAR4zGVO162EDUHyhOAqZAusu5i-6fTmy4axI-u_ydLzKqM-O_Js9MmBNrLEoryJrgBB66uvGkEFJcfezAumeAFoCPxkE2aOqSRYvzCnsZkbnuoVWtt1RPY4-DHYLP63OrtDx3JK8Mcs0YORdR20d_Pw/https%3A%2F%2Fwww.ietf.org%2Farchive%2Fid%2Fdraft-newton-regext-rdap-considerations-on-rfc9537-00.html>
HTMLized: https://datatracker.ietf.org/doc/html/draft-newton-regext-rdap-considerations-on-rfc9537<https://secure-web.cisco.com/1REPzMDdSw__QNYdJu2_kLVewC2cr1haTsVsmIarmU-cprYlCco2gFbdcSX2pPVwrfkcM-fCQxOEmba-0NcdZQ8IR0Y_-IExINWk53AiyGysA7lIGTNK7YqmMbWN_SmcpSNs7U_EhZ3fF14yDoZcqVkZHSMj0Kr8uRcPKB-IX3iArW3ZIayS2xHbCk3IxP-9otAp-K4QUd5jSUl9HByZpvm-6VrZ3dLoQtnXneeWnlpSlq0EPIiuZHgTTJTGChawd9d8MAHlybuEab0icvIWBx1HmwSiVVT-nn9ryTJCBqjM/https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-newton-regext-rdap-considerations-on-rfc9537>


Abstract:

This document discusses client implementation issues relating to RFC
9537, “Redacted Fields in the Registration Data Access Protocol
(RDAP) Response”. The considerations in this document have arisen
from problems raised by two separate teams attempting to implement
RFC 9537 in both an RDAP web client and an RDAP command line client.
Some of these problems may be insurmountable, leaving portions of RFC
9537 non-interoperable between clients and servers, while other
problems place a high degree of complexity upon clients.



The IETF Secretariat