Re: [netmod] yang-instance-file include-defaults leaf

[Andy Bierman <[e-mail]>]

On Fri, Sep 10, 2021 at 5:15 AM Balázs Lengyel <[e-mail]>
wrote:

> See below as BALAZS4
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 2021. szeptember 10., péntek 4:57
> *To:* Balázs Lengyel <[e-mail]>
> *Cc:* Rob Wilton (rwilton) <[e-mail]>; NetMod WG <[e-mail]
> >
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Thu, Sep 9, 2021 at 6:19 AM Balázs Lengyel <[e-mail]>
> wrote:
>
> Hello Andy,
>
> See below as BALAZS3.
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 2021. augusztus 25., szerda 18:29
> *To:* Balázs Lengyel <[e-mail]>
> *Cc:* Rob Wilton (rwilton) <[e-mail]>; NetMod WG <[e-mail]
> >
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi,
>
>
>
> Here is the latest text. It is inconsistent with RFC 6243, section 3.
>
> IMO the subsections should be cited instead of the copy-and-change
> approach.
>
> BALAZS3: The 6243 sections contain parts about “data retrieval”
> “capabilities” or “conceptual data nodes set by the server”
>
> These parts are not relevant for many of the instance data use cases, so I
> would like to stick with including text here.
>
>
>
>
>
>
>
> I guess I do not understand why the "with-defaults" leaf or at leaf
> "with-defaults-mode" typedef
>
> cannot be used. IMO this is bad practice.  Applications that already know
> how to deal
>
> with defaults according to RFC 6243 should be supported.
>
>
>
>
>
>      leaf includes-defaults {
>
>          type ncwd:with-defaults-mode;
>
>      }
>
>
>
> I do not see any text in this typedef that is server-specific.
>
> Andy
>
> BALAZS4:  While I generally agree that duplication is a bad practice, I
> avoided using ncwd:with-defaults-mode because:
>
> ·        It references RFC 6243 <https://datatracker.ietf.org/doc/html/rfc6243>; Section 3 <https://datatracker.ietf.org/doc/html/rfc6243#section-3> which is full of inappropriate references (server, data retrieval, capabilities)
>
> ·        The typedef’s description is "Possible modes to report default data." However, in a number of use cases (e.g., UC2 Preloading default configuration data, UC7, U8) the data is not reported.
>
> ·        report-all-tagged description contains “XML attribute” which is incorrect if we use JSON encoding
>
> ·        I prefer to use the 2119 term SHALL, SHALL NOT in the enum descriptions.
>
> Is this acceptable?
>
>
>
>

The data type I suggested does not have any references to section 3.
It does not have any MUST,SHOULD,MAY text at all.
The JSON encoding in RFC 7951 supports attributes.

IMO this leaf will cause a lot of confusion for users.
includes-defaults looks like with-defaults but it isn't.
It uses the exact same enums as with-defaults, but they mean different
things.


Andy


>
>    leaf includes-defaults {
>
>          type enumeration {
>
>            enum report-all {
>
>              value 1;
>
>              description
>
>                "All data nodes SHOULD be included independent of
>
>                  any default values.";
>
>
>
> AB: should follow 6243, 3.1
>
>            }
>
>            enum trim {
>
>              value 2;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD
>
>                  NOT be included.";
>
>
>
> AB: should follow 6243, section 3.2
>
>            }
>
>            enum explicit {
>
>              value 3;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD NOT be
>
>                  included. However, if the actual value was set by
>
>                  a NETCONF client or other management application
>
>                  by the way of an explicit management operation the
>
>                  data node SHOULD be included.";
>
>
>
> AB: should follow 6243, section 3.3
>
>            }
>
>          }
>
>          description
>
>            "As instance-data-sets MAY contain incomplete data sets,
>
>              thus any data node MAY be absent.
>
>
>
>              Providing the instance-data-set intends to contain a
>
>              full data set, this leaf specifies whether the data set
>
>              includes data nodes that have a default defined and
>
>              where the actual value is the same as the default value.
>
>
>
>              Data nodes that have no default values should always
>
>              be included.
>
>              Data nodes that have a default value, but the actual
>
>              value is not equal to the schema defined default
>
>              should always be included.";
>
>
>
>
>
> AB: The last paragraph should be removed or changed. Why are these nodes special?
>
> Nodes that are actually present and do not contain the YANG default
>
> are not relevant to this object.
>
> BALAZS3: OK
>
>
>
>          reference
>
>            "RFC 6243 <https://datatracker.ietf.org/doc/html/rfc6243>: With-defaults Capability for NETCONF";
>
>        }
>
>
>
> The best way to indicate a representation of a YANG default in the data
> set is to include
>
> the "default" attribute in each default node.
> https://datatracker.ietf.org/doc/html/rfc6243#section-6
>
> This actually works for explicit mode and leaf-lists (unlike the current
> solution).
>
> BALAZS3: OK. Added report-all-tagged enum to includes-defaults leaf.
>
> Here is the current proposed text:
>
> ------------------------------------------------------------
>
>     leaf includes-defaults {
>
>       type enumeration {
>
>         enum report-all {
>
>           value 1;
>
>           description
>
>             "All data nodes SHALL be included independent of
>
>               any default values, if the data node
>
>               is covered by the instance-data-set.";
>
>         }
>
>         enum report-all-tagged  {
>
>           value 2;
>
>           description
>
>             "All data nodes SHALL be included independent of
>
>               any default values if the data node
>
>               is covered by the instance-data-set.
>
>               Any nodes considered to be default data SHALL
>
>               contain a 'default' attribute set to 'true'";
>
>         }
>
>         enum trim {
>
>           value 3;
>
>           description
>
>             "Data nodes that have a default defined and where
>
>               the actual value is equal to the schema default
>
>               value SHALL NOT be included.";
>
>         }
>
>         enum explicit {
>
>           value 4;
>
>           description
>
>             "Data nodes where the actual value is equal to the
>
>               schema default value SHALL NOT be included.
>
>               However, if the actual value was set by a NETCONF
>
>               client or other management application by the way
>
>               of an explicit management operation, the data node
>
>               SHALL be included, if the data node is covered by
>
>               the instance-data-set.";
>
>         }
>
>       }
>
>       description
>
>         "An instance-data-set may contain or exclude default
>
>           data. This leaf indicates whether default data is
>
>           included.
>
>
>
>           As instance-data-sets MAY contain incomplete data
>
>           sets: MAY NOT cover all data nodes. A leaf or
>
>           leaf-list MAY be absent because the instance-data-set
>
>           does not intend to include the data node independent
>
>           of default handling.";
>
>       reference
>
>         "RFC 6243: With-defaults Capability for NETCONF
>
>          RFC 8040 :RESTCONF Protocol";
>
>     }
>
> ------------------------------------------------------------
>
>
>
>
>
> On Tue, Aug 24, 2021 at 1:41 AM Balázs Lengyel <
> [e-mail]> wrote:
>
> Hello Andy,
>
> In the -17 I removed the default value for includes-defaults as you
> proposed.
>
>
>
> I am not sure I understand the rest of the comments as
> instance-file-format does not use the concept of “basic-mode”. It has a
> single leaf to indicate what is the situation with defaults in the specific
> instance-data-set.
>
> As this is not a live server request/reply situation we do not want to
> specify a basic and additional modes, we just want to specify the handling
> used for this specific instance data set.
>
>
>
> Regards Balazs
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 2021. augusztus 23., hétfő 18:58
> *To:* Balázs Lengyel <[e-mail]>
> *Cc:* Rob Wilton (rwilton) <[e-mail]>; NetMod WG <[e-mail]
> >
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Mon, Aug 23, 2021 at 5:17 AM Balázs Lengyel <
> [e-mail]> wrote:
>
> Hello Rob,
>
> I think this won’t fly.
>
> In sections 1.2 and 2 we state:
>
> *“Instance data files MAY contain partial data sets.”*
>
> Which is important for many use-cases.  This means you cannot say that a default value will or must be included, as they might be omitted because they are not part of the partial data set.
>
> In a way it is difficult to separate between leaves that are missing because
>
> -        They are not part of the partial data-set
>
> -        They are omitted because they have the default value and one of the trim or explicit options is used
>
> If this becomes important the report-all options shall be used.
>
>
>
>
>
>
>
> I thought we already agreed there cannot be a default or there is no way to
>
> represent "no defaults added".
>
>
>
> Note that "report-all" is not useful if basic-mode=explicit, since a leaf
> reporting the YANG default
>
> could be set by the client.  Only report-all-tagged will clearly identify
> defaults in this case.
>
>
>
> Also note that if basic-mode=report-all then there will be no defaults
> ever reported.
>
> This mode means the server does not consider any node to be a default and
> always returns
>
> every node (if with-defaults used or not).
>
>
>
> This is the reason I used the SHOULD word.
>
> Regards Balazs
>
>
>
> Andy
>
>
>
>
>
> *From:* Rob Wilton (rwilton) <[e-mail]>
> *Sent:* 2021. augusztus 23., hétfő 12:27
> *To:* Balázs Lengyel <[e-mail]>; Andy Bierman <
> [e-mail]>; NetMod WG <[e-mail]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi Balazs, Andy, Netmod,
>
>
>
> Sorry for the delayed response.  I would still like to strength the
> description of the defaults.  E.g., RFC 6243 uses MUSTs rather than SHOULDs.
>
>
>
> Hence, I have generated some proposed alternative descriptions, that are
> somewhat stricter, but also more generically focussed only on the default
> values.
>
>
>
> With these definitions, I think that we could define the
> “include-defaults” default value to be “explicit”, since if the leaf if not
> included, then I think that this effectively what the meaning would be
> anyway.
>
>
>
>
>
> In particular, I would propose changing the descriptions as follows:
>
>
>
>        leaf includes-defaults {
>
>          type enumeration {
>
>            enum report-all {
>
>              value 1;
>
>              description
>
>                "All data nodes SHOULD be included independent of
>
>                  any default values.";
>
>            }
>
>            enum trim {
>
>              value 2;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD
>
>                  NOT be included.";
>
>            }
>
>            enum explicit {
>
>              value 3;
>
>              description
>
>                "Data nodes that have a default defined and where
>
>                  the actual value is the default value SHOULD NOT be
>
>                  included. However, if the actual value was set by
>
>                  a NETCONF client or other management application
>
>                  by the way of an explicit management operation the
>
>                  data node SHOULD be included.";
>
>            }
>
>          }
>
>
>
> Proposed:
>
>
>
>        leaf includes-defaults {
>
>          type enumeration {
>
>            enum report-all {
>
>              value 1;
>
>              description
>
>                "The instance data set includes all data nodes,
>
>                 including those that contain the schema default.”;
>
>            }
>
>            enum trim {
>
>              value 2;
>
>              description
>
>                "The instance data set excludes all data nodes
>
>                 that contain the schema default.";
>
>            }
>
>            enum explicit {
>
>              value 3;
>
>              description
>
>                "The instance data set may include some data nodes
>
>                 that match the schema default and may exclude some
>
>                 data nodes that match the schema default.”;
>
>            }
>
>          }
>
>          description
>
>            "This leaf provides an indication of how default data
>
>             is presented within an instance data set, modelled on
>
>             RFC 6243.
>
>
>
>             Interpretation of the use of defaults depends on the
>
>             context of what the instance data set represents.
>
>
>
>             E.g., if the instance data set represents configuration,
>
>             Then include-defaults aligns to the meaning of the
>
>             default-handling basic modes in RFC 6243.  If the
>
>             instance data set represents operational data from the
>
>             operational state datastore [RFC 8342], then
>
>             include-defaults aligns to the definition of that
>
>             datastore in RFC 8342.”;
>
>
>
> Would text along these lines work?
>
>
>
> Thanks,
>
> Rob
>
>
>
>
>
> *From:* Balázs Lengyel <[e-mail]>
> *Sent:* 28 July 2021 23:08
> *To:* Rob Wilton (rwilton) <[e-mail]>; Andy Bierman <
> [e-mail]>
> *Cc:* NetMod WG <[e-mail]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hello Rob,
>
> Removing the “default trim;” will address Andy’s comment.
>
>
>
> Your *in-use-values* is very specific to one of the use-cases:
> reading/documenting operational values. It is not useful for the other
> use-cases. I think the “documenting operational datastore” use-case could
> be handled by indicating the *includes-defaults=report-all*. Case (i)
> would contain the value case (ii) will not.
>
> Regards Balazs
>
>
>
> *From:* Rob Wilton (rwilton) <[e-mail]>
> *Sent:* 2021. július 27., kedd 17:38
> *To:* Andy Bierman <[e-mail]>; Balázs Lengyel <
> [e-mail]>
> *Cc:* NetMod WG <[e-mail]>
> *Subject:* RE: [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi Andy, Balazs,
>
>
>
> So, the reason that I want a flag to indicate whether default values are
> in use is because of this definition of operational in RFC 8342:
>
>
>
>    Requests to retrieve nodes from <operational> always return the value
>
>    in use if the node exists, regardless of any default value specified
>
>    in the YANG module.  If no value is returned for a given node, then
>
>    this implies that the node is not used by the device.
>
>
>
> It was written this way because otherwise a consumer of operational data
> cannot differentiate between:
>
> (i)               This value is not present because it matches the
> default value specified in the YANG module, and
>
> (ii)              This value is not present because the server has failed
> to return it for some reason (e.g., perhaps the daemon that would have
> provided this value is down or not available, or perhaps it is a bug, or
> perhaps it is not implemented and is a missing deviation).
>
>
>
> So, I think that in some cases, the absence of a data node does not
> necessarily mean that the default value is in effect, and I wanted the
> instance-data document to be able to contain and correctly report this data.
>
>
>
> I think that this behaviour could be captured by a single leaf.  Another
> way of articulating this would be:
>
>
>
> leaf in-use-values {
>
>   type boolean;
>
>   default false;
>
>   description
>
>     “Only if set to true, the absence of a value in the
>
>      instance data for a given data node implies that the
>
>     node is not used rather than implicitly taking the
>
>      default value specified by any corresponding
>
>     ‘default’ statement specified in the YANG schema.”;
>
> }
>
>
>
> With this, I’m not sure whether we need the “includes-default” leaf
> currently specified in the draft, but if we do, then I would think that
> leaf should be entirely optional, i.e., without the default “trim”.
>
>
>
> Regards,
> Rob
>
>
>
>
>
> *From:* Andy Bierman <[e-mail]>
> *Sent:* 10 July 2021 17:41
> *To:* Rob Wilton (rwilton) <[e-mail]>
> *Cc:* NetMod WG <[e-mail]>; Balázs Lengyel <
> [e-mail]>
> *Subject:* Re: [netmod] yang-instance-file include-defaults leaf
>
>
>
>
>
>
>
> On Fri, Jul 9, 2021 at 5:23 AM Rob Wilton (rwilton) <[e-mail]>
> wrote:
>
> Andy,
>
>
>
> Yes, when I suggested this, I was thinking that a boolean flag might be
> sufficient.  My point being that automatically filtering out default values
> isn’t always the right thing to do.
>
>
>
>
>
>
>
> The solution is simple.
>
> Get rid of the inappropriate "default trim" statement.
>
>
>
> If the leaf is present then it identifies the basic-mode that was used to
> include defaults.
>
> If not then the information is either not known, not applicable, or
> defaults were not added.
>
>
>
> The "default" statement is a bug because there is no default basic-mode.
>
> All of the basic-modes are in use in deployments and no camp has ever
>
> been able to convince the others that theirs is right.
>
>
>
>
>
> Andy
>
>
>
> E.g., something along these lines:
>
>
>
> leaf exclude-defaults {
>
>   type boolean;
>
>   default true;
>
>   description
>
>     “Can be used to reduce the size of the content data file.
>
>
>
>       When unset or set to true, data nodes that have a default defined and
>
>       where the actual value is the default value are excluded from the
> content
>
>       data.
>
>
>
>       When set to false, data nodes with default value are not filtered,
> and
>
>       may appear in the content data.”
>
> }
>
>
>
> Would this satisfy your concern?
>
>
>
> Regards,
> Rob
>
>
>
>
>
> *From:* netmod <[e-mail]> *On Behalf Of *Andy Bierman
> *Sent:* 08 July 2021 18:16
> *To:* NetMod WG <[e-mail]>
> *Subject:* [netmod] yang-instance-file include-defaults leaf
>
>
>
> Hi,
>
>
>
> The module has this object:
>
>
>
>     leaf includes-defaults {
>
>        type enumeration {
>
>          enum report-all {
>
>            value 1;
>
>            description
>
>              "All data nodes SHOULD be included independent of
>
>                any default values.";
>
>          }
>
>          enum trim {
>
>            value 2;
>
>            description
>
>              "Data nodes that have a default defined and where
>
>                the actual value is the default value SHOULD
>
>                NOT be included.";
>
>          }
>
>          enum explicit {
>
>            value 3;
>
>            description
>
>              "Data nodes that have a default defined and where
>
>                the actual value is the default value SHOULD NOT be
>
>                included. However, if the actual value was set by
>
>                a NETCONF client or other management application
>
>                by the way of an explicit management operation the
>
>                data node SHOULD be included.";
>
>          }
>
>        }
>
>        default trim;
>
>
>
> The draft is extremely server-centric, like most IETF standards, but this
>
> leaf is too server-centric to ignore.
>
>
>
> Consider the possibility that the source of the file is NOT a NETCONF
> server.
>
> This data may not be known so the default of "trim" may not be correct.
>
>
>
> IMO this leaf is noise because any tool that knows the schema will also
>
> know the YANG defaults.  The solution is incomplete anyway because
>
> the presence of a leaf that has a YANG default is not enough.
>
> The  "report-all-tagged" mode must be used to identify defaults.
>
> IMO this leaf should be removed, but at least add an enum called "unknown".
>
>
>
>
>
> Andy
>
>
>
>
>
>