Re: [netmod] yang-instance-file include-defaults leaf

[Andy Bierman <[e-mail]>]

On Fri, Sep 10, 2021 at 5:15 AM Balázs Lengyel <[e-mail]>
wrote:

> See below as BALAZS4
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 2021. szeptember 10., péntek 4:57
> *To:* Balázs Lengyel <[e-mail]>
> *Cc:* Rob Wilton (rwilton) <[e-mail]>; NetMod WG <[e-mail]
> >
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Thu, Sep 9, 2021 at 6:19 AM Balázs Lengyel <[e-mail]>
> wrote:
>
> Hello Andy,
>
> See below as BALAZS3.
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 2021. augusztus 25., szerda 18:29
> *To:* Balázs Lengyel <[e-mail]>
> *Cc:* Rob Wilton (rwilton) <[e-mail]>; NetMod WG <[e-mail]
> >
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi,
>
>
>
> Here is the latest text. It is inconsistent with RFC 6243, section 3.
>
> IMO the subsections should be cited instead of the copy-and-change
> approach.
>
> BALAZS3: The 6243 sections contain parts about “data retrieval”
> “capabilities” or “conceptual data nodes set by the server”
>
> These parts are not relevant for many of the instance data use cases, so I
> would like to stick with including text here.
>
>
>
>
>
>
>
> I guess I do not understand why the "with-defaults" leaf or at leaf
> "with-defaults-mode" typedef
>
> cannot be used. IMO this is bad practice.  Applications that already know
> how to deal
>
> with defaults according to RFC 6243 should be supported.
>
>
>
>
>
>      leaf includes-defaults {
>
>          type ncwd:with-defaults-mode;
>
>      }
>
>
>
> I do not see any text in this typedef that is server-specific.
>
> Andy
>
> BALAZS4:  While I generally agree that duplication is a bad practice, I
> avoided using ncwd:with-defaults-mode because:
>
> ·        It references RFC 6243 <https://datatracker.ietf.org/doc/html/rfc6243>; Section 3 <https://datatracker.ietf.org/doc/html/rfc6243#section-3> which is full of inappropriate references (server, data retrieval, capabilities)
>
> ·        The typedef’s description is "Possible modes to report default data." However, in a number of use cases (e.g., UC2 Preloading default configuration data, UC7, U8) the data is not reported.
>
> ·        report-all-tagged description contains “XML attribute” which is incorrect if we use JSON encoding
>
> ·        I prefer to use the 2119 term SHALL, SHALL NOT in the enum descriptions.
>
> Is this acceptable?
>
>
>
>


The <get-data> operation uses the with-defaults parameter directly.

https://datatracker.ietf.org/doc/html/rfc8526#section-3.1.1.2

IMO this operation should be consistent with <get-data>.


Andy


>
>    leaf includes-defaults {
>
>          type enumeration {
>
>            enum report-all {
>
>              value 1;
>
>              description
>
>                "All data nodes SHOULD be included independent of
>
>                  any default values.";
>
>
>
> AB: should follow 6243, 3.1
>
>            }
>
>            enum trim {
>
>              value 2;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD
>
>                  NOT be included.";
>
>
>
> AB: should follow 6243, section 3.2
>
>            }
>
>            enum explicit {
>
>              value 3;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD NOT be
>
>                  included. However, if the actual value was set by
>
>                  a NETCONF client or other management application
>
>                  by the way of an explicit management operation the
>
>                  data node SHOULD be included.";
>
>
>
> AB: should follow 6243, section 3.3
>
>            }
>
>          }
>
>          description
>
>            "As instance-data-sets MAY contain incomplete data sets,
>
>              thus any data node MAY be absent.
>
>
>
>              Providing the instance-data-set intends to contain a
>
>              full data set, this leaf specifies whether the data set
>
>              includes data nodes that have a default defined and
>
>              where the actual value is the same as the default value.
>
>
>
>              Data nodes that have no default values should always
>
>              be included.
>
>              Data nodes that have a default value, but the actual
>
>              value is not equal to the schema defined default
>
>              should always be included.";
>
>
>
>
>
> AB: The last paragraph should be removed or changed. Why are these nodes special?
>
> Nodes that are actually present and do not contain the YANG default
>
> are not relevant to this object.
>
> BALAZS3: OK
>
>
>
>          reference
>
>            "RFC 6243 <https://datatracker.ietf.org/doc/html/rfc6243>: With-defaults Capability for NETCONF";
>
>        }
>
>
>
> The best way to indicate a representation of a YANG default in the data
> set is to include
>
> the "default" attribute in each default node.
> https://datatracker.ietf.org/doc/html/rfc6243#section-6
>
> This actually works for explicit mode and leaf-lists (unlike the current
> solution).
>
> BALAZS3: OK. Added report-all-tagged enum to includes-defaults leaf.
>
> Here is the current proposed text:
>
> ------------------------------------------------------------
>
>     leaf includes-defaults {
>
>       type enumeration {
>
>         enum report-all {
>
>           value 1;
>
>           description
>
>             "All data nodes SHALL be included independent of
>
>               any default values, if the data node
>
>               is covered by the instance-data-set.";
>
>         }
>
>         enum report-all-tagged  {
>
>           value 2;
>
>           description
>
>             "All data nodes SHALL be included independent of
>
>               any default values if the data node
>
>               is covered by the instance-data-set.
>
>               Any nodes considered to be default data SHALL
>
>               contain a 'default' attribute set to 'true'";
>
>         }
>
>         enum trim {
>
>           value 3;
>
>           description
>
>             "Data nodes that have a default defined and where
>
>               the actual value is equal to the schema default
>
>               value SHALL NOT be included.";
>
>         }
>
>         enum explicit {
>
>           value 4;
>
>           description
>
>             "Data nodes where the actual value is equal to the
>
>               schema default value SHALL NOT be included.
>
>               However, if the actual value was set by a NETCONF
>
>               client or other management application by the way
>
>               of an explicit management operation, the data node
>
>               SHALL be included, if the data node is covered by
>
>               the instance-data-set.";
>
>         }
>
>       }
>
>       description
>
>         "An instance-data-set may contain or exclude default
>
>           data. This leaf indicates whether default data is
>
>           included.
>
>
>
>           As instance-data-sets MAY contain incomplete data
>
>           sets: MAY NOT cover all data nodes. A leaf or
>
>           leaf-list MAY be absent because the instance-data-set
>
>           does not intend to include the data node independent
>
>           of default handling.";
>
>       reference
>
>         "RFC 6243: With-defaults Capability for NETCONF
>
>          RFC 8040 :RESTCONF Protocol";
>
>     }
>
> ------------------------------------------------------------
>
>
>
>
>
> On Tue, Aug 24, 2021 at 1:41 AM Balázs Lengyel <
> [e-mail]> wrote:
>
> Hello Andy,
>
> In the -17 I removed the default value for includes-defaults as you
> proposed.
>
>
>
> I am not sure I understand the rest of the comments as
> instance-file-format does not use the concept of “basic-mode”. It has a
> single leaf to indicate what is the situation with defaults in the specific
> instance-data-set.
>
> As this is not a live server request/reply situation we do not want to
> specify a basic and additional modes, we just want to specify the handling
> used for this specific instance data set.
>
>
>
> Regards Balazs
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 2021. augusztus 23., hétfő 18:58
> *To:* Balázs Lengyel <[e-mail]>
> *Cc:* Rob Wilton (rwilton) <[e-mail]>; NetMod WG <[e-mail]
> >
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Mon, Aug 23, 2021 at 5:17 AM Balázs Lengyel <
> [e-mail]> wrote:
>
> Hello Rob,
>
> I think this won’t fly.
>
> In sections 1.2 and 2 we state:
>
> *“Instance data files MAY contain partial data sets.”*
>
> Which is important for many use-cases.  This means you cannot say that a default value will or must be included, as they might be omitted because they are not part of the partial data set.
>
> In a way it is difficult to separate between leaves that are missing because
>
> -        They are not part of the partial data-set
>
> -        They are omitted because they have the default value and one of the trim or explicit options is used
>
> If this becomes important the report-all options shall be used.
>
>
>
>
>
>
>
> I thought we already agreed there cannot be a default or there is no way to
>
> represent "no defaults added".
>
>
>
> Note that "report-all" is not useful if basic-mode=explicit, since a leaf
> reporting the YANG default
>
> could be set by the client.  Only report-all-tagged will clearly identify
> defaults in this case.
>
>
>
> Also note that if basic-mode=report-all then there will be no defaults
> ever reported.
>
> This mode means the server does not consider any node to be a default and
> always returns
>
> every node (if with-defaults used or not).
>
>
>
> This is the reason I used the SHOULD word.
>
> Regards Balazs
>
>
>
> Andy
>
>
>
>
>
> *From:* Rob Wilton (rwilton) <[e-mail]>
> *Sent:* 2021. augusztus 23., hétfő 12:27
> *To:* Balázs Lengyel <[e-mail]>; Andy Bierman <
> [e-mail]>; NetMod WG <[e-mail]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi Balazs, Andy, Netmod,
>
>
>
> Sorry for the delayed response.  I would still like to strength the
> description of the defaults.  E.g., RFC 6243 uses MUSTs rather than SHOULDs.
>
>
>
> Hence, I have generated some proposed alternative descriptions, that are
> somewhat stricter, but also more generically focussed only on the default
> values.
>
>
>
> With these definitions, I think that we could define the
> “include-defaults” default value to be “explicit”, since if the leaf if not
> included, then I think that this effectively what the meaning would be
> anyway.
>
>
>
>
>
> In particular, I would propose changing the descriptions as follows:
>
>
>
>        leaf includes-defaults {
>
>          type enumeration {
>
>            enum report-all {
>
>              value 1;
>
>              description
>
>                "All data nodes SHOULD be included independent of
>
>                  any default values.";
>
>            }
>
>            enum trim {
>
>              value 2;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD
>
>                  NOT be included.";
>
>            }
>
>            enum explicit {
>
>              value 3;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD NOT be
>
>                  included. However, if the actual value was set by
>
>                  a NETCONF client or other management application
>
>                  by the way of an explicit management operation the
>
>                  data node SHOULD be included.";
>
>            }
>
>          }
>
>
>
> Proposed:
>
>
>
>        leaf includes-defaults {
>
>          type enumeration {
>
>            enum report-all {
>
>              value 1;
>
>              description
>
>                "The instance data set includes all data nodes,
>
>                 including those that contain the schema default.”;
>
>            }
>
>            enum trim {
>
>              value 2;
>
>              description
>
>                "The instance data set excludes all data nodes
>
>                 that contain the schema default.";
>
>            }
>
>            enum explicit {
>
>              value 3;
>
>              description
>
>                "The instance data set may include some data nodes
>
>                 that match the schema default and may exclude some
>
>                 data nodes that match the schema default.”;
>
>            }
>
>          }
>
>          description
>
>            "This leaf provides an indication of how default data
>
>             is presented within an instance data set, modelled on
>
>             RFC 6243.
>
>
>
>             Interpretation of the use of defaults depends on the
>
>             context of what the instance data set represents.
>
>
>
>             E.g., if the instance data set represents configuration,
>
>             Then include-defaults aligns to the meaning of the
>
>             default-handling basic modes in RFC 6243.  If the
>
>             instance data set represents operational data from the
>
>             operational state datastore [RFC 8342], then
>
>             include-defaults aligns to the definition of that
>
>             datastore in RFC 8342.”;
>
>
>
> Would text along these lines work?
>
>
>
> Thanks,
>
> Rob
>
>
>
>
>
> *From:* Balázs Lengyel <[e-mail]>
> *Sent:* 28 July 2021 23:08
> *To:* Rob Wilton (rwilton) <[e-mail]>; Andy Bierman <
> [e-mail]>
> *Cc:* NetMod WG <[e-mail]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hello Rob,
>
> Removing the “default trim;” will address Andy’s comment.
>
>
>
> Your *in-use-values* is very specific to one of the use-cases:
> reading/documenting operational values. It is not useful for the other
> use-cases. I think the “documenting operational datastore” use-case could
> be handled by indicating the *includes-defaults=report-all*. Case (i)
> would contain the value case (ii) will not.
>
> Regards Balazs
>
>
>
> *From:* Rob Wilton (rwilton) <[e-mail]>
> *Sent:* 2021. július 27., kedd 17:38
> *To:* Andy Bierman <[e-mail]>; Balázs Lengyel <
> [e-mail]>
> *Cc:* NetMod WG <[e-mail]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi Andy, Balazs,
>
>
>
> So, the reason that I want a flag to indicate whether default values are
> in use is because of this definition of operational in RFC 8342:
>
>
>
>    Requests to retrieve nodes from <operational> always return the value
>
>    in use if the node exists, regardless of any default value specified
>
>    in the YANG module.  If no value is returned for a given node, then
>
>    this implies that the node is not used by the device.
>
>
>
> It was written this way because otherwise a consumer of operational data
> cannot differentiate between:
>
> (i)               This value is not present because it matches the
> default value specified in the YANG module, and
>
> (ii)              This value is not present because the server has failed
> to return it for some reason (e.g., perhaps the daemon that would have
> provided this value is down or not available, or perhaps it is a bug, or
> perhaps it is not implemented and is a missing deviation).
>
>
>
> So, I think that in some cases, the absence of a data node does not
> necessarily mean that the default value is in effect, and I wanted the
> instance-data document to be able to contain and correctly report this data.
>
>
>
> I think that this behaviour could be captured by a single leaf.  Another
> way of articulating this would be:
>
>
>
> leaf in-use-values {
>
>   type boolean;
>
>   default false;
>
>   description
>
>     “Only if set to true, the absence of a value in the
>
>      instance data for a given data node implies that the
>
>     node is not used rather than implicitly taking the
>
>      default value specified by any corresponding
>
>     ‘default’ statement specified in the YANG schema.”;
>
> }
>
>
>
> With this, I’m not sure whether we need the “includes-default” leaf
> currently specified in the draft, but if we do, then I would think that
> leaf should be entirely optional, i.e., without the default “trim”.
>
>
>
> Regards,
> Rob
>
>
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 10 July 2021 17:41
> *To:* Rob Wilton (rwilton) <[e-mail]>
> *Cc:* NetMod WG <[e-mail]>; Balázs Lengyel <
> [e-mail]>
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Fri, Jul 9, 2021 at 5:23 AM Rob Wilton (rwilton) <[e-mail]>
> wrote:
>
> Andy,
>
>
>
> Yes, when I suggested this, I was thinking that a boolean flag might be
> sufficient.  My point being that automatically filtering out default values
> isn’t always the right thing to do.
>
>
>
>
>
>
>
> The solution is simple.
>
> Get rid of the inappropriate "default trim" statement.
>
>
>
> If the leaf is present then it identifies the basic-mode that was used to
> include defaults.
>
> If not then the information is either not known, not applicable, or
> defaults were not added.
>
>
>
> The "default" statement is a bug because there is no default basic-mode.
>
> All of the basic-modes are in use in deployments and no camp has ever
>
> been able to convince the others that theirs is right.
>
>
>
>
>
> Andy
>
>
>
> E.g., something along these lines:
>
>
>
> leaf exclude-defaults {
>
>   type boolean;
>
>   default true;
>
>   description
>
>     “Can be used to reduce the size of the content data file.
>
>
>
>       When unset or set to true, data nodes that have a default defined and
>
>       where the actual value is the default value are excluded from the
> content
>
>       data.
>
>
>
>       When set to false, data nodes with default value are not filtered,
> and
>
>       may appear in the content data.”
>
> }
>
>
>
> Would this satisfy your concern?
>
>
>
> Regards,
> Rob
>
>
>
>
>
> *From:* netmod <[e-mail]> *On Behalf Of *Andy Bierman
> *Sent:* 08 July 2021 18:16
> *To:* NetMod WG <[e-mail]>
> *Subject:* [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi,
>
>
>
> The module has this object:
>
>
>
>     leaf includes-defaults {
>
>        type enumeration {
>
>          enum report-all {
>
>            value 1;
>
>            description
>
>              "All data nodes SHOULD be included independent of
>
>                any default values.";
>
>          }
>
>          enum trim {
>
>            value 2;
>
>            description
>
>              "Data nodes that have a default defined and where
>
>                the actual value is the default value SHOULD
>
>                NOT be included.";
>
>          }
>
>          enum explicit {
>
>            value 3;
>
>            description
>
>              "Data nodes that have a default defined and where
>
>                the actual value is the default value SHOULD NOT be
>
>                included. However, if the actual value was set by
>
>                a NETCONF client or other management application
>
>                by the way of an explicit management operation the
>
>                data node SHOULD be included.";
>
>          }
>
>        }
>
>        default trim;
>
>
>
> The draft is extremely server-centric, like most IETF standards, but this
>
> leaf is too server-centric to ignore.
>
>
>
> Consider the possibility that the source of the file is NOT a NETCONF
> server.
>
> This data may not be known so the default of "trim" may not be correct.
>
>
>
> IMO this leaf is noise because any tool that knows the schema will also
>
> know the YANG defaults.  The solution is incomplete anyway because
>
> the presence of a leaf that has a YANG default is not enough.
>
> The  "report-all-tagged" mode must be used to identify defaults.
>
> IMO this leaf should be removed, but at least add an enum called "unknown".
>
>
>
>
>
> Andy
>
>
>
>
>
>