Re: [netmod] OpsState Direction Impact on Recommended IETF YANG Model Structure

Andy Bierman <andy@yumaworks.com> Fri, 29 July 2016 16:13 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B5B4612D7EE for <netmod@ietfa.amsl.com>; Fri, 29 Jul 2016 09:13:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yumaworks-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mYVCfpdYwVYt for <netmod@ietfa.amsl.com>; Fri, 29 Jul 2016 09:13:10 -0700 (PDT)
Received: from mail-ua0-x22a.google.com (mail-ua0-x22a.google.com [IPv6:2607:f8b0:400c:c08::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8E5AF12D66B for <netmod@ietf.org>; Fri, 29 Jul 2016 09:13:10 -0700 (PDT)
Received: by mail-ua0-x22a.google.com with SMTP id l32so65157197ual.2 for <netmod@ietf.org>; Fri, 29 Jul 2016 09:13:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=03wAgMwxP1mWyZEtPzKs7596PVp6BIImhfPtylp19qM=; b=zHbl2xEOYP6YLopL2m5q5gyXx73uEni7qNZQgIRvBT6JDVc/MLKWhK4iVY++okV1SL e8CIGUxqjIqnTFsuZyelmXztM+k7AeQOUONURf4zRullaJgEf9g4+Ruq3uQ4GDnEOlyv anO5gIgmKIP8i0cGQ+nTV9/Eqn9wRKvhpA+wAtR16LhCDaSwGmLnKdEdvIfXxGBAXW2N BqXB91GwOanZNDLfTkTwKJuU6prmkM18QDUPGDMcAuBSkylTsKaONsQ9HzDMb1QZtfKa vIi9E85ZEo0tLfa0COA4O+6KdzGPsqRMtQ59PYdI9kHuVkXTkAqldOeITDDOIiFRumK4 W7kw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=03wAgMwxP1mWyZEtPzKs7596PVp6BIImhfPtylp19qM=; b=Zkv+qeICf3tdtbvEEeFoNHxR5eFdQ0vvgo1+IUFsLjEG7zp//NNnZr0XENzFaqKPNM aDVbowzDXUPOVx9llJ8E+SLCZU4BdERy0grK9fkMa0cPXlEOOGZlmBAt1CwD5hHUPtmF 3hsCh+kNRWdjou9hx8PnsDewRjReO6XPaDj50aSMMoYaGMzZ14sYvWhQeEvtKrV5ljUI IU6uEebcZvrhBOK2eN+pU+JRwL2vljcc/ZTcUcv+Av2+UMLWVy44qnqqMlMVasdtDLWU 0d3cJEYelhQNGYqM4v78/4AVuruDErOKRS3XnDgV8uEv6grvSBjWlPRaYfgiA4x/DJvf 0A+Q==
X-Gm-Message-State: AEkooutQtFQJlbwmTLpydl1lq3vuSNH0EWXe1MmmmPV5Y43vLpMBQhbL1X88/W+vjpYRc3AXXyCtVOvbmPNbnA==
X-Received: by 10.176.67.4 with SMTP id k4mr19130680uak.47.1469808789404; Fri, 29 Jul 2016 09:13:09 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.103.4.198 with HTTP; Fri, 29 Jul 2016 09:13:08 -0700 (PDT)
In-Reply-To: <e307c2fc-e621-b5c6-4fcc-67bbcdcb87b9@cisco.com>
References: <D3A935F0.6A4DC%acee@cisco.com> <eb15fd23-2c0a-50c4-1ebc-7c0e4867dfd8@cisco.com> <20160721174033.GB54646@elstar.local> <d18f5dd0-64d0-e223-88a9-faa4df4b7866@cisco.com> <DCB3EBBF-5EB1-4C8E-AA55-F59C4B5A8E4D@juniper.net> <bed9398c-0e6a-450e-d2ac-b381b6bebf87@cisco.com> <5296754B-8178-4B1B-B4A6-FE228ABB8E7F@juniper.net> <9367f4b1-7814-e175-32e8-d518438b841d@cisco.com> <m24m79c1ja.fsf@birdie.labs.nic.cz> <D3BF8708.72620%acee@cisco.com> <552008CB-F216-4578-A709-AE0613C2EFB9@nic.cz> <12ed1a4d-44c5-9a51-b6d4-95e1620a24ee@cisco.com> <m260roedim.fsf@birdie.labs.nic.cz> <CABCOCHRF98sTa=MA1VTkZovOufL-=yQr9Gzy+ojnncjSfK1y0Q@mail.gmail.com> <e307c2fc-e621-b5c6-4fcc-67bbcdcb87b9@cisco.com>
From: Andy Bierman <andy@yumaworks.com>
Date: Fri, 29 Jul 2016 09:13:08 -0700
Message-ID: <CABCOCHQnJPR34u+z+0SJLyQEb7Q0TS4oN4W6fOL5a7Exk6wEvQ@mail.gmail.com>
To: Robert Wilton <rwilton@cisco.com>
Content-Type: multipart/alternative; boundary="94eb2c09507ac7d7ae0538c886ad"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/wQU8k0BdPDSLD4dmUIyKFIprN8o>
Cc: netmod WG <netmod@ietf.org>
Subject: Re: [netmod] OpsState Direction Impact on Recommended IETF YANG Model Structure
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Jul 2016 16:13:13 -0000

Hi,

You said "automated code".
Write some YANG extensions for your tools to manually assign
the mapping.  This has already been suggested several times.
This approach can also address N:M mappings.  Supporting 1:1
mappings and ignoring everything else doesn't really help.


Andy


On Fri, Jul 29, 2016 at 8:35 AM, Robert Wilton <rwilton@cisco.com> wrote:

> Hi Andy,
>
> The main problem that I'm currently trying to solve, and get agreement on,
> is "where does the operator's automated code look to find the state nodes
> associated with a particular config node".
>
> E.g. for feature /foo, do they look for the config false nodes under /foo,
> or do the look somewhere in /foo-state.  The current answer appears to be,
> that the config false nodes could appear in either place in the IETF
> modules without any requirement for a consistent approach.  Is that right?
>
> Separately, the OpenConfig folks have said that it would be better if they
> could get (or register for notification of) all of the data associated with
> "foo" with a single request rather than having to make multiple requests in
> different trees.  Personally, I prefer their model where they have merged
> interface-state into interfaces.  Mainly because I think that it should
> make the YANG modules easier to read with less duplication of structure,
> and I think that it is a better long term direction to go in.  However, I
> appreciate Acee's comments that we may just be too late to change this now,
> and we must just accept the status quo (whatever that is).
>
> I would like to know what should the common approach for IETF standard
> models be?  E.g. is it one of the following:
>
> 1) All config false leaves for foo must go under /foo-state.
> 2) All config false leaves for foo must go under /foo
> 3) All config false leaves go under /foo where possible, or /foo-state
> otherwise (e.g. for restconf-monitoring).
> 4) Config false leaves go wherever the model writer decides is appropriate.
>
> I've put further comments inline below ...
>
>
> On 29/07/2016 15:39, Andy Bierman wrote:
>
> Hi,
>
> I am somewhat confused about this discussion.
> Apparently it is a hyge problem to put foo-counters under
> foo-state?
>
> RW:
> This isn't just about counters, it is all config false nodes that are not
> present solely to represent "applied configuration".
>
>   Configuration must be used (and setup by the operator?)
> in order for foo-counters to exist?
>
> RW:
> No, this is not being proposed.
>
> draft-schoenw and draft-wilton propose that the config true nodes are
> allowed to exist in an "operational state datastore" to parent child config
> false nodes, even though they haven't been configured.  I also proposed
> that metadata annotations could be used to indicate that the containing
> config=true nodes in the operational state datastore exist only in the
> system (and not in the configuration).
>
> Further I would propose that NETCONF/RESTCONF servers could support an
> "operational state datastore" without any requirement to distinguish
> applied configuration separately from intended configuration.
>
>
> So what problem does this solve?
>
> RW:
> I see that it potentially allows modules to avoid an arbitrary foo vs
> foo-state split.
>
>
> The opstate solution proposal requires a config path-expr to be altered
> to generate the corresponding path-expr under the 'state' container.
>
> RW:
> Are you referring to the OpenConfig model solution here or draft-schoenw,
> or draft-wilton?
>
> It does not seem to be a problem that a new path-expr needs to be
> constructed.
> Is that what you are trying to solve? Even though counters never have
> a corresponding configuration node with the same name?
>
>
> So what happens to modules like ietf-restconf-monitoring?
>
> RW:
> Either it stays exactly as it is now, or alternatively it could just be
> named "restconf" rather than "restconf-state", even if it just contains
> config false nodes.
>
> It is no longer allowed to write monitoring modules?
>
> RW:
> It is still allowed to write monitoring modules, no change here.
>
> They all have to contained within some configuration nodes?
>
> RW:
> No, not necessarily:
>  - it they are just state then either call them "foo" or "foo-state" as
> per above.
>  - if they are mix of config and state then they would go under "foo",
> which would be a config true node (but not necessarily configured, as per
> my comments on the operational state datastore above).
>
> Again, what problem is this trying to solve?
>
> RW:
> Hopefully this one has already been answered above.
>
> Thanks,
> Rob
>
>
>
>
> Andy
>
>
> On Fri, Jul 29, 2016 at 7:15 AM, Ladislav Lhotka <lhotka@nic.cz> wrote:
>
>> Robert Wilton <rwilton@cisco.com> writes:
>>
>> > On 28/07/2016 15:20, Ladislav Lhotka wrote:
>> >>> On 28 Jul 2016, at 15:57, Acee Lindem (acee) <acee@cisco.com> wrote:
>> >>>
>> >>> Hi Lada,
>> >>>
>> >>> On 7/28/16, 9:52 AM, "netmod on behalf of Ladislav Lhotka"
>> >>> <netmod-bounces@ietf.org on behalf of lhotka@nic.cz> wrote:
>> >>>
>> >>>> Robert Wilton <rwilton@cisco.com> writes:
>> >>>>
>> >>>>> On 26/07/2016 21:36, Kent Watsen wrote:
>> >>>>>> <Rob Wilton writes>
>> >>>>>>
>> >>>>>>
>> >>>>>> So my thinking is that if we can't merge "foo-state" into "foo"
>> then
>> >>>>>> instead we should have consistent rules that explicitly state that
>> for
>> >>>>>> all IETF models "foo" and "foo-state" are separate trees with a
>> >>>>>> consistent naming convention and structure.  That should hopefully
>> >>>>>> allow tooling to programmatically relate the two separate trees
>> >>>>>> together.  It may give a path to allow "foo-state" to be merged
>> into
>> >>>>>> "foo" in future, but once IETF has standardized 600+ models with
>> >>>>>> separate sub-trees, I cannot see that they would get merged back
>> >>>>>> together again.
>> >>>>>>
>> >>>>>> What other alternatives are available?  As a WG we need to tell the
>> >>>>>> other WGs how the IETF YANG models should be structured.
>> >>>>>>
>> >>>>>> In short, unfortunately I think that we have probably already
>> missed
>> >>>>>> the opportunity to merge "foo" and "foo-state" subtrees together
>> ...
>> >>>>>>
>> >>>>>> </Rob Wilton>
>> >>>>>>
>> >>>>>> Firstly, I’m trying to get a sense of how big a problem this
>> >>>>>> foo/foo-state thing is.  [Note: by foo-state, I’m only referring to
>> >>>>>> counters, not opstate].
>> >>>>>>
>> >>>>> RW:
>> >>>>> By counters, I think that we also mean any config false nodes that
>> >>>>> don't
>> >>>>> directly represent "applied configuration", right?  E.g. is an
>> >>>>> interface
>> >>>>> operationally up or down.
>> >>>>>
>> >>>>>>    I know about RFC 7223, which was done out of consideration for
>> >>>>>> system-generated interfaces, but how many other such models are
>> there
>> >>>>>> envisioned to be?
>> >>>>>>
>> >>>>> RW:
>> >>>>> - Any models that augment RFC 7223 and have config false nodes will
>> be
>> >>>>> impacted.
>> >>>>> - I thought that quite a lot of other IETF models that are in the
>> >>>>> process of being standardized have a top level split between "foo"
>> and
>> >>>>> "foo-state".  E.g the ISIS model (draft-ietf-isis-yang-isis-cfg-08)
>> has
>> >>>>> this split.  I suspect that all the routing models will be
>> structured
>> >>>>> similarly.
>> >>>> Correct. One reason is that the core routing model envisions
>> >>>> system-controlled RIBs.
>> >>>>
>> >>>>> - Although it is perhaps worth pointing out that I think that the
>> >>>>> OpenConfig modules effectively have exactly this same issue (e.g.
>> they
>> >>>>> have a combined interfaces tree keyed by config true leaves), and
>> they
>> >>>>> pragmatically just ignore the issue of system created interface
>> >>>>> entries.
>> >>>> The NETMOD WG considered this issue quite important in the past.
>> >>>>
>> >>>> My impression from the OpState discussion is that we are on the
>> quest of
>> >>>> the philosopher's stone, trying to find a shortcut where none is
>> >>>> possible in general. The long session in Berlin concentrated on the
>> >>>> life-cycle of a single parameter that's somehow configured, then
>> >>>> manipulated, and eventually ends up as operational state. IMO this
>> >>>> is too simplistic, the relationship between configuration and state
>> can
>> >>>> be much more complex. RIB is one example - it combines contributions
>> >>>> from configuration (static routes) and derived state (routing
>> >>>> protocols).
>> >>> If one were to support the Applied-Config data store, it be comprised
>> of
>> >>> only the current state of the configured static routes.  The complete
>> RIB
>> >>> would still need to be accessible in separate data nodes.
>> >> Yes, but I didn't talk about intended-applied. I understand that
>> another goal of OpState is to unify config and (true) state and get rid of
>> the foo and foo-state dichotomy in the data model. I am sceptical about it.
>> > The goal is/was to unify where the only reason that they were split was
>> > because the lifetime of the configured containing datanode may differ
>> > from the operational containing datanode.  E.g. interfaces vs
>> > interfaces-state was split to allow for system created interfaces that
>> > were not configured, but other than this reason the split seems quite
>> > artificial and not particularly helpful.
>> >
>> > OpenConfig is modelling interfaces and interfaces-state as a single
>> > list.  It would be kind of helpful if IETF models and OpenConfig models
>> > could be consistent in this regard, and I prefer the combined list
>> > approach used by OpenConfig interfaces (on the assumption that we can
>> > solve the technical problems associated with this approach - which I
>> > think that we can).
>>
>> If we agree that there may be state data that have no direct
>> counterpart in configuration, and these will be in foo-state, then the
>> likely outcome is that in order to collect all state data, one will have
>> to query both foo and foo-state, which doesn't seem ideal.
>>
>> Lada
>>
>> >
>> > I've no particular issue with a RIB existing under routing-state. But
>> > personally, if it was the ISIS specific routing table, I would prefer it
>> > to be under a single top level ISIS container on the assumption that you
>> > cannot really have an ISIS routing table if ISIS isn't actually running
>> > on the device.
>> >
>> > Cheers,
>> > Rob
>> >
>> >>
>> >> Lada
>> >>
>> >>> Thanks,
>> >>> Acee
>> >>>
>> >>>
>> >>>
>> >>>> After all, most real devices have some configuration mode and "show"
>> >>>> commands. They are separate even though there is certainly some
>> >>>> relationship between their data.
>> >>>>
>> >>>> Lada
>> >>>>
>> >>>>>> Is this issue currently blocking models from progressing, or are we
>> >>>>>> getting ourselves wrapped around a hypothetical?
>> >>>>>>
>> >>>>> RW:
>> >>>>> I think that it is blocking models from progressing.
>> >>>>>
>> >>>>> The current guidance for "intended vs applied" is clear.  I.e. there
>> >>>>> must not be "config false" leaves in the IETF YANG data models to
>> >>>>> represent "applied config".
>> >>>>>
>> >>>>> But there is no clear guidance for the rest of operational state
>> that
>> >>>>> isn't applied config.  The other WGs need clear guidance
>> (effectively
>> >>>>> now) to ensure that they can start publishing models as RFCs.
>> >>>>>
>> >>>>>
>> >>>>>>   If RFC 7223 is an outlier, then we can address it as a special
>> case
>> >>>>>> (perhaps via the related-state/related-config YANG annotations).
>> What
>> >>>>>> do you think?
>> >>>>>>
>> >>>>> RW:
>> >>>>> Personally, I would like one common convention that applies to all
>> IETF
>> >>>>> YANG models.
>> >>>>>
>> >>>>> Idealistically I would like foo and foo-state to be merged because I
>> >>>>> think that will make the models easier to use and maintain in the
>> long
>> >>>>> term, but I don't know if we are just too late to go in that
>> direction.
>> >>>>>
>> >>>>> It seems to me that the NETMOD WG really should try to come to a
>> >>>>> decision quite quickly on this, but I don't know how to encourage
>> that.
>> >>>>>
>> >>>>> A virtual interim on just this topic perhaps?
>> >>>>>
>> >>>>>> Next, regarding paths forward (assuming 7223 is not an outlier),
>> I’m
>> >>>>>> thinking the opposite.  I’m quite sure that we would never merge
>> the
>> >>>>>> 600+ models with separate subtrees back together again.  So I’m
>> >>>>>> thinking we immediately merge foo and foo-state in all active YANG
>> >>>>>> models (so that the YANG “conceptual” models are stable and good)
>> >>>>>> *and* then we use your idea to programmatically generate the
>> >>>>>> “foo-state” tree, presumably only when needed.  This foo-state tree
>> >>>>>> could be generated offline by tools and provided as a second YANG
>> >>>>>> module in drafts.  In this way, servers (opstate aware or not) can
>> >>>>>> advertise if clients can access the foo-state tree (an
>> opstate-aware
>> >>>>>> server may still advertise it for business reasons, and it can
>> >>>>>> ‘deprecate’ the tree when no longer needed).   We could do the same
>> >>>>>> without tools today by just using a feature statement on, for
>> >>>>>> instance, the interfaces-state container, but I like pushing for
>> >>>>>> tooling upfront so that we’re guaranteed mergeability later.
>> Thoughts?
>> >>>>>>
>> >>>>> RW:
>> >>>>> So the generated "foo-state" tree would contain a copy of all config
>> >>>>> false nodes in the YANG schema and a "config false copy" of any
>> config
>> >>>>> true nodes in the YANG schema that are required to provide parental
>> >>>>> structure for the descendant config false nodes.
>> >>>>> - The Xpath expressions would also need to be adjusted, and possibly
>> >>>>> some of those might break (or need to be fixed by hand).
>> >>>>> - Groupings might be a problem, but potentially they could be
>> expanded.
>> >>>>>
>> >>>>> Technically this solution might work, but is it possible to get
>> >>>>> everyone
>> >>>>> to agree that this is the right direction to go in before we spend
>> time
>> >>>>> on this?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Rob
>> >>>>>
>> >>>>>
>> >>>>>> Kent // as a contributor
>> >>>>>>
>> >>>>> _______________________________________________
>> >>>>> netmod mailing list
>> >>>>> netmod@ietf.org
>> >>>>> https://www.ietf.org/mailman/listinfo/netmod
>> >>>> --
>> >>>> Ladislav Lhotka, CZ.NIC Labs
>> >>>> PGP Key ID: E74E8C0C
>> >>>>
>> >>>> _______________________________________________
>> >>>> netmod mailing list
>> >>>> netmod@ietf.org
>> >>>> https://www.ietf.org/mailman/listinfo/netmod
>> >> --
>> >> Ladislav Lhotka, CZ.NIC Labs
>> >> PGP Key ID: E74E8C0C
>> >>
>> >>
>> >>
>> >>
>> >> .
>> >>
>> >
>>
>> --
>> Ladislav Lhotka, CZ.NIC Labs
>> PGP Key ID: E74E8C0C
>>
>> _______________________________________________
>> netmod mailing list
>> netmod@ietf.org
>> https://www.ietf.org/mailman/listinfo/netmod
>>
>
>
>