Re: [netmod] xpath expressions in JSON

"Reshad Rahman (rrahman)" <rrahman@cisco.com> Fri, 12 October 2018 17:34 UTC

Return-Path: <rrahman@cisco.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8683F130E60 for <netmod@ietfa.amsl.com>; Fri, 12 Oct 2018 10:34:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.957
X-Spam-Level:
X-Spam-Status: No, score=-14.957 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.456, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ezfBIEHl7IfB for <netmod@ietfa.amsl.com>; Fri, 12 Oct 2018 10:34:36 -0700 (PDT)
Received: from alln-iport-6.cisco.com (alln-iport-6.cisco.com [173.37.142.93]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7F15C130E1A for <netmod@ietf.org>; Fri, 12 Oct 2018 10:34:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=15444; q=dns/txt; s=iport; t=1539365676; x=1540575276; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=0YOXsFVo1GE2Fog1AJVbnxQuhDoftdrz3ctm+6+/5LQ=; b=IFm4YRk3W2+iKj+Bg7QInizFD4uEIAADu6qsCvcjHJdGr4FIOg7wgW+6 oCYZggJYjkyIs9w0Jl5S+CCgkbbHcdroCh4T17vlZb+iYpwMX5g5FerwA CDHAp3yfNi63UKRQGfYgZN4pXn3BRcx/aWiL8IZfJmTclNsm77XQtw8VH k=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A0ADAAAQ2sBb/4YNJK1aChkBAQEBAQEBAQEBAQEHAQEBAQEBgVEEAQEBAQELAYFUL2Z/KAqDa4gWjDGBaCWXB4F6CwEBGAsJg3pGAheERSE0DQ0BAwEBAgEBAm0cDIU5AQEBAwEBASEEDToLEAIBCA4KAgImAgICJQsVEAIECgQFgyABgXkID6VJezOJUwWBC4o7F4FBP4ESJwwTgkyDGwEBgTZCI4JHMYImAohhhVmFdYkaUwkCkFYXgU+Eb4MRhkeVfQIRFIEmHTiBVXAVOyoBgkGLGIU+b4EoihQBgR4BAQ
X-IronPort-AV: E=Sophos;i="5.54,373,1534809600"; d="scan'208";a="184541545"
Received: from alln-core-12.cisco.com ([173.36.13.134]) by alln-iport-6.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 12 Oct 2018 17:34:35 +0000
Received: from XCH-ALN-008.cisco.com (xch-aln-008.cisco.com [173.36.7.18]) by alln-core-12.cisco.com (8.15.2/8.15.2) with ESMTPS id w9CHYZNi022979 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Fri, 12 Oct 2018 17:34:35 GMT
Received: from xch-rcd-005.cisco.com (173.37.102.15) by XCH-ALN-008.cisco.com (173.36.7.18) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Fri, 12 Oct 2018 12:34:34 -0500
Received: from xch-rcd-005.cisco.com ([173.37.102.15]) by XCH-RCD-005.cisco.com ([173.37.102.15]) with mapi id 15.00.1395.000; Fri, 12 Oct 2018 12:34:34 -0500
From: "Reshad Rahman (rrahman)" <rrahman@cisco.com>
To: Martin Bjorklund <mbj@tail-f.com>, "Robert Wilton -X (rwilton - ENSOFT LIMITED at Cisco)" <rwilton@cisco.com>
CC: "netmod@ietf.org" <netmod@ietf.org>
Thread-Topic: [netmod] xpath expressions in JSON
Thread-Index: AQHUYIqyJUA4hM8yQEuWRAlaWHhia6UYv5+AgAAV+YCAAIZOgIAASaoAgABHYACAAC1hgIAAEM2AgAAGGwCAAAHbgIAAB16AgABShQCAAAlqgIABCfeAgABTBAA=
Date: Fri, 12 Oct 2018 17:34:34 +0000
Message-ID: <09F70090-6876-462C-B2C4-0319CEAA10BC@cisco.com>
References: <7a9b1a5f-db66-7431-a2c7-82acebb2082b@cisco.com> <20181011.181150.1840683183107627057.mbj@tail-f.com> <329f67ff-0774-0fd7-f630-7a01c7e7d3be@cisco.com> <20181012.103727.731509761734796510.mbj@tail-f.com>
In-Reply-To: <20181012.103727.731509761734796510.mbj@tail-f.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.b.0.180311
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [161.44.212.54]
Content-Type: text/plain; charset="utf-8"
Content-ID: <726F579F0CEA044BB5E9C388DCEFE4E0@emea.cisco.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Outbound-SMTP-Client: 173.36.7.18, xch-aln-008.cisco.com
X-Outbound-Node: alln-core-12.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/FT32UZuVFOidqV893qjinQp2CYw>
Subject: Re: [netmod] xpath expressions in JSON
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Oct 2018 17:34:40 -0000

On 2018-10-12, 4:37 AM, "netmod on behalf of Martin Bjorklund" <netmod-bounces@ietf.org on behalf of mbj@tail-f.com> wrote:

    Robert Wilton <rwilton@cisco.com> wrote:
    > 
    > 
    > On 11/10/2018 17:11, Martin Bjorklund wrote:
    > > Robert Wilton <rwilton@cisco.com> wrote:
    > >>
    > >> On 11/10/2018 11:50, Martin Bjorklund wrote:
    > >>> Robert Wilton <rwilton@cisco.com> wrote:
    > >>>> On 11/10/2018 11:21, Martin Bjorklund wrote:
    > >>>>> Andy Bierman <andy@yumaworks.com> wrote:
    > >>>>>> On Wed, Oct 10, 2018 at 11:39 PM, Martin Bjorklund <mbj@tail-f.com>
    > >>>>>> wrote:
    > >>>>>>
    > >>>>>>> Andy Bierman <andy@yumaworks.com> wrote:
    > >>>>>>>> On Wed, Oct 10, 2018 at 6:59 PM, Reshad Rahman (rrahman) <
    > >>>>>>> rrahman@cisco.com>
    > >>>>>>>> wrote:
    > >>>>>>>>
    > >>>>>>>>> On 2018-10-10, 9:59 AM, "netmod on behalf of Martin Bjorklund" <
    > >>>>>>>>> netmod-bounces@ietf.org on behalf of mbj@tail-f.com> wrote:
    > >>>>>>>>>
    > >>>>>>>>>        Ladislav Lhotka <lhotka@nic.cz> wrote:
    > >>>>>>>>>        > Martin Bjorklund <mbj@tail-f.com> writes:
    > >>>>>>>>>        >
    > >>>>>>>>>        > > Hi,
    > >>>>>>>>>        > >
    > >>>>>>>>>        > > While reviewing restconf-notif, I saw this example:
    > >>>>>>>>>        > >
    > >>>>>>>>>        > >    {
    > >>>>>>>>>        > >       "ietf-subscribed-notifications:input": {
    > >>>>>>>>>        > >          "stream": "NETCONF",
    > >>>>>>>>>        > >          "stream-xpath-filter": "/ds:foo/",
    > >>>>>>>>>        > >          "dscp": "10"
    > >>>>>>>>>        > >       }
    > >>>>>>>>>        > >    }
    > >>>>>>>>>        > >
    > >>>>>>>>>        > > Note the "stream-xpath-filter".  It has a prefix in the
    > >>>>>>>>>        > > XPath
    > >>>>>>>>> string.
    > >>>>>>>>>        > > How are prefixes declared when JSON is used?
    > >>>>>>>>>        > >
    > >>>>>>>>>        > > The leaf "stream-xpath-filter" says:
    > >>>>>>>>>        > >
    > >>>>>>>>>        > >               o The set of namespace declarations are
    > >>>>>>>>>        > >               those
    > >>>>>>>>>        > >               in
    > >>>>>>>>> scope on
    > >>>>>>>>>        > >                  the 'stream-xpath-filter' leaf element.
    > >>>>>>>>>        > >
    > >>>>>>>>>        > > (I think I provided that text...)
    > >>>>>>>>>        > >
    > >>>>>>>>>        > > This assumes that the encoding is XML, or at leas that the
    > >>>>>>> encoding
    > >>>>>>>>>        > > can somehow transfer namespace declarations.
    > >>>>>>>>>        >
    > >>>>>>>>>        > It can't. There are two options:
    > >>>>>>>>>        >
    > >>>>>>>>>        > 1. have different representations of this value in XML and
    > >>>>>>>>>        > JSON,
    > >>>>>>>>>        >    analogically to instance indentifiers (sec. 6.11 in RFC
    > >>>>>>>>>        >    7951).
    > >>>>>>>>>        >
    > >>>>>>>>>        > 2. use a module name rather than a prefix in XML, too.
    > >>>>>>>>>        >
    > >>>>>>>>>        > I would suggest #2.
    > >>>>>>>>> <RR> But that means making non-backwards compatible change to the XML
    > >>>>>>>>> representation?
    > >>>>>>>>>
    > >>>>>>>> Not really. It means NETMOD WG would be creating its own special
    > >>>>>>>> variant
    > >>>>>>> of
    > >>>>>>>> XPath.
    > >>>>>>> Not at all.  What I propose is perfectly fine, legal XPath 1.0.
    > >>>>>>>
    > >>>>>>> XPath 1.0 says that an XPath expression is evaluated in a context.
    > >>>>>>> One item in the context is a set of mappings from <prefix> to <uri>,
    > >>>>>>> where <prefix> is used to lookup prefixes used in the XPath
    > >>>>>>> expression, e.g. in "/foo:interfaces" "foo" is the prefix.
    > >>>>>>>
    > >>>>>>> It is perfectly fine to say that the prefix mapping set is this:
    > >>>>>>>
    > >>>>>>>       "ietf-interfaces" ->
    > >>>>>>>       "urn:ietf:params:xml:ns:yang:ietf-interfaces"
    > >>>>>>>       "ietf-ip"         -> "urn:ietf:params:xml:ns:yang:ietf-ip"
    > >>>>>>>
    > >>>>>>> and use that to evaluate the expression
    > >>>>>>>
    > >>>>>>>      /ietf-interfaces:interfaces/ietf-interfaces:interface/ietf-ip/ipv4
    > >>>>>>>
    > >>>>>>>
    > >>>>>>>
    > >>>>>> The XPath expression is normally parsed within an XML instance
    > >>>>>> document.
    > >>>>>> There are "xmlns" attributes present that map the prefix to a
    > >>>>>> namespace URI.
    > >>>>>> These mappings will not be present in the JSON at all.
    > >>>>>>
    > >>>>>> A custom XPath implementation is required to magically identify the
    > >>>>>> prefix
    > >>>>>> as a module name and magically find the namespace URI for the module
    > >>>>>> name.
    > >>>>> I disagree.  You need an XPath implementation + custom code to set up
    > >>>>> the environment.
    > >>>> This is OK, but can we just use the JSON encoding instance identifier
    > >>>> format exactly?  I.e .RFC 7951 section 6.11.
    > >>>>
    > >>>> So "/ietf-interfaces:interfaces/interface/ietf-ip:ipv4/enabled"
    > >>>>
    > >>>> can trivially be expanded to:
    > >>>>
    > >>>> "/ietf-interfaces:interfaces/ietf-interfaces:interface/ietf-ip:ipv4/ietf-ip:enabled",
    > >>>>
    > >>>> and then interpreted with the context:
    > >>>>        "ietf-interfaces" -> "urn:ietf:params:xml:ns:yang:ietf-interfaces"
    > >>>>      "ietf-ip"         -> "urn:ietf:params:xml:ns:yang:ietf-ip"
    > >>> *this* would require a custom XPath implementation.
    > >> Why?  I.e. how is this different from stating "Custom code is needed
    > >> to connect things together"?
    > > B/c the specification of XPath allows you to (actually, *requires* you
    > > to) construct the set of prefix strings to url mappings.
    > >
    > > This is "custom code to connect things".
    > >
    > > But changing the syntax means changin the specification.
    > Not really.
    > 
    > It would just mean that the filter value is not an "Xpath"
    > expression.  It is a more a concise string that can be expanded into
    > an Xpath expression.
    > 
    > >
    > >>> and it is not obvious what the rules for the "auto-assignment" of
    > >>> prefixes would be.  For example:
    > >>>
    > >>>     /ietf-interfaces//ietf-ip:address[../foo]
    > >>>
    > >>> what is the prefix for "foo"?
    > >> OK, so here the module for "../foo" would need to be specified.
    > >>
    > >> Perhaps the rule that I'm looking for is the module name may be
    > >> omitted when it matches the parent node module, and can easily be
    > >> inferred.  I.e. so that for any XPath string, it is possible to
    > >> trivially expand it without any additional schema context.
    > >>
    > >> It just seems to be that requiring the long hand of
    > >> "/ietf-interfaces:interfaces/ietf-interfaces:interface/ietf-ip:ipv4/ietf-ip:enabled"
    > >> seems like it will get very verbose, and I wonder whether we are
    > >> introducing yet another Xpath format to YANG.
    > > I agree that it is very verbose.  But do not mix XPath expressions in
    > > leaf values (which is what this thread is about) with
    > > instance-identfiers.
    > OK, but ultimately:
    > - these are both leaf values.
    > - they both identify nodes in a YANG datastore.
    > - the fact that their format is somewhat subtlety different will catch
    >   people out.
    
    Agreed.
    
    We already have:
    
      o  instance-identifier in XML uses prefixes from the XML document
      o  instance-identifier in JSON uses module names as prefixes
      o  XPath in NETCONF filter uses prefixes from the XML document
      o  XPath in JSON query filter uses module names as prefixes
    
    So an alternative could be to use different encodings for
    "stream-xpath-filter" as well, depending on if it is XML or JSON.
    
    So, we could do in SN:
    
                 o  If the node is encoded in XML, the set of namespace
                    declarations are those in scope on the
                    'stream-xpath-filter' leaf element.
    
                 o  If the node is encoded in JSON, the set of namespace
                    declarations is the set of prefix and namespace pairs
                    for all supported YANG modules, where the prefix is
                    the YANG module name and the namespace is as defined
                    by the "namespace" statement in the YANG module.

    This is far from perfect, but at least the format is consistent within
    each encoding.  
<RR> I'd be ok with that since it's consistent with what's already done for other "types". So the stream-xpath-filter in A.1.1 of RESTCONF-notif would be along the lines of "/ex-foo:foo/".


One problem is that it is unclear how to handle other
    encodings.
    
    
    If we do this, we'll set a precedence for future models.
    
    Now, suppose we have a module "ex-foo" with namespace
    "urn:example:foo".  If an XML client writes:
    
       <example-expr xmlns:f="urn:example:foo">
         /f:bar/f:baz
       </example-expr>
    
    a JSON will read this node as:
    
       "example-expr": "/ex-foo:bar/ex-foo:baz"
    
    
    [This context-dependent encoding that we have for a couple of types is
    probably the biggest mistake in YANG]
    
    
    BTW, we already have this issue w/ NACM's node-instance-identifier.
    It is highly unclear what a JSON client is supposed to see if it reads
    NACM rules with node-instance-identifiers.
    
    > >> Finally, I'm trying to figure out have RFC 8040 query parameter (sect
    > >> 4.8.4), which also uses XPath expressions is meant to work. That
    > >> states:
    > >>
    > >> The set of namespace declarations is the set of prefix and
    > >>        namespace pairs for all supported YANG modules, where the prefix
    > >>        is the YANG module name and the namespace is as defined by the
    > >>        "namespace" statement in the YANG module.
    > > Perfect!  It seems the authors of 8040 thought of this ;-)
    > OK, what you propose would at least be consistent with how the XPath
    > is formed in sec 8040, 4.8.4?
    > 
    > I can live with that.  But still strongly think that WG should think
    > of trying to move YANG on from Xpath 1.0.
    > 
    > >
    > >> Yet the examples in section 8.3.6 don't seem to use namespace prefixes
    > >> in very many places, e.g. why is it "/example-mod:event1/name='joe'"
    > >> and not "/example-mod:event1/example-mod:name='joe'"?  Is the example
    > >> wrong, or otherwise what am I missing? :-)
    > > It seems the example is wrong!
    > Please can you check section 8040, 8.3.6.  Are all the examples wrong?
    
    Yes it seems so.  Except the last one.
<RR> I'm volunteering Rob to submit the errata (

Regards,
Reshad.    
    
    /martin
    
    _______________________________________________
    netmod mailing list
    netmod@ietf.org
    https://www.ietf.org/mailman/listinfo/netmod