Re: [Jsonpath] Robert Wilton's Discuss on draft-ietf-jsonpath-base-19: (with DISCUSS and COMMENT)

[Greg Dennis <[e-mail]>]

 The implicit cast doesn't have to do with the exists() function, but with the implications of what @.foo would mean if not an existence test.
In your "strict mode" which requires the use of exists() for existence testing, $[@.foo] would consider whether "foo" contains a "truthy" value.  (It would have to be truthiness of some kind, whether that means any "non-empty"/non-zero value or only specifically JSON "true".)  We explicitly opted against truthiness.  We didn't have functions or the type system when we made that decision, but in hindsight, truthiness is basically an implicit cast from a ValueType to a LogicalType.  (My experience, across multiple JSON extensions, is that trying to deal with truthiness just results in multiple headaches.)
More importantly, though, having an option on the implementation means that the same path could operate two different ways, which complicates determinism.  $[@.foo] in strict mode returns a different result from the same path in non-strict mode.  I think this would be frustrating for users.  Carsten addressed this with
> My knee jerk reaction is that this splits JSONPath up a bit, into the strict version and a non-strict version, adding the need to constantly think about which version one addresses etc.
Greg
    On Thursday, September 28, 2023 at 03:17:46 AM GMT+13, Rob Wilton (rwilton) <[e-mail]> wrote:  
 
  
Hi Greg,
 
  
 
I’m not sure that I follow.  I think that the only extra bit that would need to defined is the exists() function call.  Wouldn’t this end up being defined as something:

exists() Function Extension:
 
  
 
Parameters:
    
   - Nodelist
 
  
 
Result: LogicalType
 
Returns LogicalFalse if the nodelist parameter is empty, or LogicalFalse otherwise.  The function is an explicit function equivalent to the implicit behaviour defined in section 2.3.5.2.1.
 
  
 
What is the implicit cast that would need to be defined?
 
  
 
Regards,
 
Rob
 
  
 
  
 
  
 
  
 
  
 
From: Greg Dennis <[e-mail]>
Sent: Thursday, September 21, 2023 11:05 PM
To: Rob Wilton (rwilton) <[e-mail]>; Carsten Bormann <[e-mail]>
Cc: The IESG <[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: [Jsonpath] Robert Wilton's Discuss on draft-ietf-jsonpath-base-19: (with DISCUSS and COMMENT)
 
  
 
> This would mean that tools could have a switch to enable or require a strict mode that requires the explicit exists() calls, or smart text editors could optionally put in the implicit strict() function call conversion.
 
  
 
It would also mean that we would have to define implicit casts from ValueType to LogicalType (which we have already explicitly decided against) to support "strict mode" evaluation of a path that contains implicit existence checks.
 
  
 
Greg
 
  
 
On Friday, September 22, 2023 at 09:13:28 AM GMT+12, Carsten Bormann <[e-mail]> wrote:
 
  
 
  
 
Hi Rob,
 
  
 
Thank you for clearing the DISCUSS, and apologies for being mostly offline for another week.
 
  
 
>From the remaining items below, we have removed items that we think you indicated are covered.
 
  
 
> On 8. Sep 2023, at 16:17, Rob Wilton (rwilton) <rwilton=[e-mail]> wrote:
 
>
 
> Hi Carsten, all,
 
>
 
> Sorry for the delay, your response collided with a week of PTO and then the IESG telechat.
 
>
 
> I will clear my discuss based on your proposed resolution below, but still have a couple of comments that I think that it would be good to discuss a bit further please.
 
>
 
> Please see inline …
 
>
 
> […]
 
>
 
>>> […]
 
>>> (1) p 12, sec 2.1.  Overview
 
>>>
 
>>>  Specifically, if a valid JSONPath query is evaluated
 
>>> against a structured value whose size does not fit in the range of
 
>>> exact values, interfering with the correct interpretation of the
 
>>> query, the implementation MUST provide an indication of overflow.
 
>>>
 
>>> I found this paragraph to be not particularly clear.  Is this saying that using
 
>>> a value larger than 2^53 as an integer will cause problems?
 
>>
 
>> Now in PR #508:
 
>>
 
>>https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/508
 
>>
 
>> We hope this is now more readable.
 
> [Rob Wilton (rwilton)]
 
>
 
> This change resolves my discuss comment, but as a further comment (that you are free to ignore if you wish) possibly "exact integer values" might be even clearer and more precise.
 
  
 
Indeed.  Added section reference to the I-JSON reference, too.
 
  
 
Now in PR #510:
 
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/510
 
  
 
>>> ----------------------------------------------------------------------
 
>>> COMMENT:
 
>>> ----------------------------------------------------------------------
 
>>>
 
>>> Minor level comments:
 
>>>
 
>>> (4) p 10, sec 1.5.  JSONPath Examples
 
>>>
 
>>> Table 2 shows some JSONPath queries that might be applied to this
 
>>> example and their intended results.
 
>>>
 
>>> Here, and in the table title below, it wasn't clear to me why it is "intended
 
>>> results" rather than just "results".
 
>>
 
>> Well, the result of the query is not "the authors of all books in the store”, but
 
>> probably more like
 
>>
 
>> "Nigel Rees"
 
>> "Evelyn Waugh"
 
>> "Herman Melville"
 
>> "J. R. R. Tolkien”
 
>>
 
>> So the “intended” was supposed to describe the intention that might be
 
>> behind this query, not the actual query result.
 
> [Rob Wilton (rwilton)]
 
>
 
> Okay.  I had sort of read "intended results" as "hopefully your implementation returns something like", rather than the "your implementation must return something like", but it doesn't really matter.
 
  
 
We are happy with the way this is being said now.
 
  
 
>>> (7) p 18, sec 2.3.1.3.  Examples
 
>>>
 
>>>                  Table 5: Name selector examples
 
>>>
 
>>> Various table list "Result Paths", but I presume that these are only intended
 
>>> to help make the specification clear, and are not returned (by a standard
 
>>> implementation).  Would this be worth stating or clarifying?
 
>>
 
>> 2.1.2 Semantics (in 2.1 Overview) says about query results:
 
>>
 
>>>> Depending on the specific API, it might be presented as an array of the JSON
 
>> values at the nodes, an array of Normalized Paths referencing the nodes, or
 
>> both — or some other representation as desired by the implementation. Note:
 
>> an empty nodelist is a valid query result.
 
> [Rob Wilton (rwilton)]
 
>
 
> Ah, so this is an interesting comment, because I didn't get that from when reading the specification.  When I read the specification, I got the impression that it is only the values that are returned and not the paths.  Hence, please consider whether adding text to help clarify this would be worthwhile.
 
  
 
We believe this is covered in #section-2.1.2-4 [1], the first place that discusses how JSONPath query results might be presented in a specific API.
 
  
 
[1]:https://www.ietf.org/archive/id/draft-ietf-jsonpath-base-20.html#section-2.1.2-4
 
  
 
  
 
>>> (11) p 30, sec 2.3.5.2.1.  Existence Tests
 
>>>
 
>>> *  they work with queries that select structured values.
 
>>> To examine the value of a node selected by a query, an explicit
 
>>> comparison is necessary.  For example, to test whether the node
 
>>> selected by the query @.foo has the value null, use @.foo == null
 
>>> (see Section 2.6) rather than the negated existence test!@.foo
 
>>> (which yields false if @.foo selects a node, regardless of the node's
 
>>> value).
 
>>>
 
>>> I presume that there is no way to change this, but from a correctness
 
>>> perspective, I wish that the existance check had to be explicit, (e.g., perhaps
 
>>> by a function call),
 
>>
 
>> The WG discussed several different ways to skin this cat, and what you see is
 
>> the result that slowly emerged.
 
>> We could replace existence tests with a function extension such as
 
>> exists(NodesType) that works similar to count(NodesType), but returns a
 
>> LogicalType.
 
>> People would then probably complain about the verbosity, now having to
 
>> write $.a[?exists(@.b)] instead of $.a[?@.b]
 
> [Rob Wilton (rwilton)]
 
>
 
> One suggestion here could to be define the exists() function, but still allow the existing implicit exists check.  This would mean that tools could have a switch to enable or require a strict mode that requires the explicit exists() calls, or smart text editors could optionally put in the implicit strict() function call conversion.
 
>
 
> Would the authors/WG be amenable to that?  Note - this isn't part of m discuss (because I hadn't raised it), but I think that this would be a good thing to do.
 
  
 
My knee jerk reaction is that this splits JSONPath up a bit, into the strict version and a non-strict version, adding the need to constantly think about which version one addresses etc.  So I think this would indeed require discussion in the WG.  I think that, since this function can easily be added later (function extensions are an extension point with a registry), we don’t have to complete such a discussion before moving forward with the document.
 
  
 
>>> to avoid the likely common mistake you describe above,
 
>>> particularly if the test is against a boolean value.  Possibly giving the
 
>>> example for the boolean value case may also be helpful to be explicit here.
 
>>
 
>> A bit of an example is already in the text you cited above.
 
> [Rob Wilton (rwilton)]
 
>
 
> I was actually meaning to give an example for the problem case where the behaviour is not intuitive.
 
>
 
> E.g., in many programming languages the expressions "if (x) ..." and "if (x == true) ... " mean the same thing, but my understanding is that inboth  XPath and JsonPath they don't, and hence having an example to highlight the necessity of an explicit boolean check will probably be helpful to folks surprised by the behaviour.
 
  
 
Not sure the example isn’t detracting a bit from the flow of 2.3.5.2.1, but I have added a short sentence to the end of #section-2.3.5.2.1-4
 
  
 
Now in PR #511:
 
https://github.com/ietf-wg-jsonpath/draft-ietf-jsonpath-base/pull/511
 
  
 
Grüße, Carsten
 
  
 
--
 
JSONpath mailing list
 
[e-mail]
 
https://www.ietf.org/mailman/listinfo/jsonpath