Re: [netmod] xpath expressions in JSON

[Robert Wilton <[e-mail]>]

On 12/10/2018 09:37, Martin Bjorklund wrote:
> Robert Wilton <[e-mail]> wrote:
>>
>> On 11/10/2018 17:11, Martin Bjorklund wrote:
>>> Robert Wilton <[e-mail]> wrote:
>>>> On 11/10/2018 11:50, Martin Bjorklund wrote:
>>>>> Robert Wilton <[e-mail]> wrote:
>>>>>> On 11/10/2018 11:21, Martin Bjorklund wrote:
>>>>>>> Andy Bierman <[e-mail]> wrote:
>>>>>>>> On Wed, Oct 10, 2018 at 11:39 PM, Martin Bjorklund <[e-mail]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Andy Bierman <[e-mail]> wrote:
>>>>>>>>>> On Wed, Oct 10, 2018 at 6:59 PM, Reshad Rahman (rrahman) <
>>>>>>>>> [e-mail]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> On 2018-10-10, 9:59 AM, "netmod on behalf of Martin Bjorklund" <
>>>>>>>>>>> [e-mail] on behalf of [e-mail]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>         Ladislav Lhotka <[e-mail]> wrote:
>>>>>>>>>>>         > Martin Bjorklund <[e-mail]> writes:
>>>>>>>>>>>         >
>>>>>>>>>>>         > > Hi,
>>>>>>>>>>>         > >
>>>>>>>>>>>         > > While reviewing restconf-notif, I saw this example:
>>>>>>>>>>>         > >
>>>>>>>>>>>         > >    {
>>>>>>>>>>>         > >       "ietf-subscribed-notifications:input": {
>>>>>>>>>>>         > >          "stream": "NETCONF",
>>>>>>>>>>>         > >          "stream-xpath-filter": "/ds:foo/",
>>>>>>>>>>>         > >          "dscp": "10"
>>>>>>>>>>>         > >       }
>>>>>>>>>>>         > >    }
>>>>>>>>>>>         > >
>>>>>>>>>>>         > > Note the "stream-xpath-filter".  It has a prefix in the
>>>>>>>>>>>         > > XPath
>>>>>>>>>>> string.
>>>>>>>>>>>         > > How are prefixes declared when JSON is used?
>>>>>>>>>>>         > >
>>>>>>>>>>>         > > The leaf "stream-xpath-filter" says:
>>>>>>>>>>>         > >
>>>>>>>>>>>         > >               o The set of namespace declarations are
>>>>>>>>>>>         > >               those
>>>>>>>>>>>         > >               in
>>>>>>>>>>> scope on
>>>>>>>>>>>         > >                  the 'stream-xpath-filter' leaf element.
>>>>>>>>>>>         > >
>>>>>>>>>>>         > > (I think I provided that text...)
>>>>>>>>>>>         > >
>>>>>>>>>>>         > > This assumes that the encoding is XML, or at leas that the
>>>>>>>>> encoding
>>>>>>>>>>>         > > can somehow transfer namespace declarations.
>>>>>>>>>>>         >
>>>>>>>>>>>         > It can't. There are two options:
>>>>>>>>>>>         >
>>>>>>>>>>>         > 1. have different representations of this value in XML and
>>>>>>>>>>>         > JSON,
>>>>>>>>>>>         >    analogically to instance indentifiers (sec. 6.11 in RFC
>>>>>>>>>>>         >    7951).
>>>>>>>>>>>         >
>>>>>>>>>>>         > 2. use a module name rather than a prefix in XML, too.
>>>>>>>>>>>         >
>>>>>>>>>>>         > I would suggest #2.
>>>>>>>>>>> <RR> But that means making non-backwards compatible change to the XML
>>>>>>>>>>> representation?
>>>>>>>>>>>
>>>>>>>>>> Not really. It means NETMOD WG would be creating its own special
>>>>>>>>>> variant
>>>>>>>>> of
>>>>>>>>>> XPath.
>>>>>>>>> Not at all.  What I propose is perfectly fine, legal XPath 1.0.
>>>>>>>>>
>>>>>>>>> XPath 1.0 says that an XPath expression is evaluated in a context.
>>>>>>>>> One item in the context is a set of mappings from <prefix> to <uri>,
>>>>>>>>> where <prefix> is used to lookup prefixes used in the XPath
>>>>>>>>> expression, e.g. in "/foo:interfaces" "foo" is the prefix.
>>>>>>>>>
>>>>>>>>> It is perfectly fine to say that the prefix mapping set is this:
>>>>>>>>>
>>>>>>>>>        "ietf-interfaces" ->
>>>>>>>>>        "urn:ietf:params:xml:ns:yang:ietf-interfaces"
>>>>>>>>>        "ietf-ip"         -> "urn:ietf:params:xml:ns:yang:ietf-ip"
>>>>>>>>>
>>>>>>>>> and use that to evaluate the expression
>>>>>>>>>
>>>>>>>>>       /ietf-interfaces:interfaces/ietf-interfaces:interface/ietf-ip/ipv4
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> The XPath expression is normally parsed within an XML instance
>>>>>>>> document.
>>>>>>>> There are "xmlns" attributes present that map the prefix to a
>>>>>>>> namespace URI.
>>>>>>>> These mappings will not be present in the JSON at all.
>>>>>>>>
>>>>>>>> A custom XPath implementation is required to magically identify the
>>>>>>>> prefix
>>>>>>>> as a module name and magically find the namespace URI for the module
>>>>>>>> name.
>>>>>>> I disagree.  You need an XPath implementation + custom code to set up
>>>>>>> the environment.
>>>>>> This is OK, but can we just use the JSON encoding instance identifier
>>>>>> format exactly?  I.e .RFC 7951 section 6.11.
>>>>>>
>>>>>> So "/ietf-interfaces:interfaces/interface/ietf-ip:ipv4/enabled"
>>>>>>
>>>>>> can trivially be expanded to:
>>>>>>
>>>>>> "/ietf-interfaces:interfaces/ietf-interfaces:interface/ietf-ip:ipv4/ietf-ip:enabled",
>>>>>>
>>>>>> and then interpreted with the context:
>>>>>>         "ietf-interfaces" -> "urn:ietf:params:xml:ns:yang:ietf-interfaces"
>>>>>>       "ietf-ip"         -> "urn:ietf:params:xml:ns:yang:ietf-ip"
>>>>> *this* would require a custom XPath implementation.
>>>> Why?  I.e. how is this different from stating "Custom code is needed
>>>> to connect things together"?
>>> B/c the specification of XPath allows you to (actually, *requires* you
>>> to) construct the set of prefix strings to url mappings.
>>>
>>> This is "custom code to connect things".
>>>
>>> But changing the syntax means changin the specification.
>> Not really.
>>
>> It would just mean that the filter value is not an "Xpath"
>> expression.  It is a more a concise string that can be expanded into
>> an Xpath expression.
>>
>>>>> and it is not obvious what the rules for the "auto-assignment" of
>>>>> prefixes would be.  For example:
>>>>>
>>>>>      /ietf-interfaces//ietf-ip:address[../foo]
>>>>>
>>>>> what is the prefix for "foo"?
>>>> OK, so here the module for "../foo" would need to be specified.
>>>>
>>>> Perhaps the rule that I'm looking for is the module name may be
>>>> omitted when it matches the parent node module, and can easily be
>>>> inferred.  I.e. so that for any XPath string, it is possible to
>>>> trivially expand it without any additional schema context.
>>>>
>>>> It just seems to be that requiring the long hand of
>>>> "/ietf-interfaces:interfaces/ietf-interfaces:interface/ietf-ip:ipv4/ietf-ip:enabled"
>>>> seems like it will get very verbose, and I wonder whether we are
>>>> introducing yet another Xpath format to YANG.
>>> I agree that it is very verbose.  But do not mix XPath expressions in
>>> leaf values (which is what this thread is about) with
>>> instance-identfiers.
>> OK, but ultimately:
>> - these are both leaf values.
>> - they both identify nodes in a YANG datastore.
>> - the fact that their format is somewhat subtlety different will catch
>>    people out.
> Agreed.
>
> We already have:
>
>    o  instance-identifier in XML uses prefixes from the XML document
>    o  instance-identifier in JSON uses module names as prefixes
>    o  XPath in NETCONF filter uses prefixes from the XML document
>    o  XPath in JSON query filter uses module names as prefixes
Yes, but in the two JSON cases, the way that they use the module name 
prefixes is still different.  I.e. they differ in the required namespace 
encoding when a child element is in the same namespace it is parent, for 
an instance-identifier they MUST be excluded, for the XPath in JSON 
query filter they MUST be included.

I think that the crux of the problem is XPath is designed/specified for 
XML documents.

>
> So an alternative could be to use different encodings for
> "stream-xpath-filter" as well, depending on if it is XML or JSON.
>
> So, we could do in SN:
>
>               o  If the node is encoded in XML, the set of namespace
>                  declarations are those in scope on the
>                  'stream-xpath-filter' leaf element.
>
>               o  If the node is encoded in JSON, the set of namespace
>                  declarations is the set of prefix and namespace pairs
>                  for all supported YANG modules, where the prefix is
>                  the YANG module name and the namespace is as defined
>                  by the "namespace" statement in the YANG module.
>
> This is far from perfect, but at least the format is consistent within
> each encoding.  One problem is that it is unclear how to handle other
> encodings.
I agree that this is a valid alternative.  I don't know whether or not I 
prefer it.  Arguably it is more consistent and doesn't cause the RFC 
7951 JSON style encoding to leech into the XML encoding.

I would expect the CBOR encoding to either define its own binary format 
for this, or follow the JSON style.

>
>
> If we do this, we'll set a precedence for future models.
Yes, this is the part that concerns me the most, that we are reinforcing 
that the format for these type of leaves can be encoding dependent.

>
> Now, suppose we have a module "ex-foo" with namespace
> "urn:example:foo".  If an XML client writes:
>
>     <example-expr xmlns:f="urn:example:foo">
>       /f:bar/f:baz
>     </example-expr>
>
> a JSON will read this node as:
>
>     "example-expr": "/ex-foo:bar/ex-foo:baz"
>
>
> [This context-dependent encoding that we have for a couple of types is
> probably the biggest mistake in YANG]
I agree that it is far from ideal.

In an ideal world, I would like to see an encoding like the JSON 
instance identifier one (because it seems reasonably concise and simple) 
used for both instance identifiers and XPath expressions, but that would 
require us defining a replacement for XPath and is obviously out of 
scope for this particular problem.

>
>
> BTW, we already have this issue w/ NACM's node-instance-identifier.
> It is highly unclear what a JSON client is supposed to see if it reads
> NACM rules with node-instance-identifiers.
I agree.

Thanks,
Rob


>
>>>> Finally, I'm trying to figure out have RFC 8040 query parameter (sect
>>>> 4.8.4), which also uses XPath expressions is meant to work. That
>>>> states:
>>>>
>>>> The set of namespace declarations is the set of prefix and
>>>>         namespace pairs for all supported YANG modules, where the prefix
>>>>         is the YANG module name and the namespace is as defined by the
>>>>         "namespace" statement in the YANG module.
>>> Perfect!  It seems the authors of 8040 thought of this ;-)
>> OK, what you propose would at least be consistent with how the XPath
>> is formed in sec 8040, 4.8.4?
>>
>> I can live with that.  But still strongly think that WG should think
>> of trying to move YANG on from Xpath 1.0.
>>
>>>> Yet the examples in section 8.3.6 don't seem to use namespace prefixes
>>>> in very many places, e.g. why is it "/example-mod:event1/name='joe'"
>>>> and not "/example-mod:event1/example-mod:name='joe'"?  Is the example
>>>> wrong, or otherwise what am I missing? :-)
>>> It seems the example is wrong!
>> Please can you check section 8040, 8.3.6.  Are all the examples wrong?
> Yes it seems so.  Except the last one.
>
>
> /martin
> .
>