Re: [netmod] Comments on NMDA-04

[Robert Wilton <[e-mail]>]

Hi Andy,


On 29/09/2017 17:46, Andy Bierman wrote:
>
>
> On Thu, Sep 28, 2017 at 9:28 AM, Robert Wilton <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
>     Hi,
>
>     Regarding the issue "Is it allowed to violate uniqueness of key
>     values?", https://github.com/netmod-wg/datastore-dt/issues/10
>     <https://github.com/netmod-wg/datastore-dt/issues/10>
>
>     We have discussed this further, and would like to extend the text
>     in the draft to explicitly allow key uniqueness to be violated!
>
>
> so we can rubber-stamp buggy servers?
> I do not agree this is needed.

So, the aim here is not to standardize buggy servers.

The aim here is to ensure that clients understand that the content of 
the operational state datastore aims to reflect the live operational 
state of the device.  Because it is reflecting live data, and that data 
may be constantly changing, it is not possibly to guarantee that the 
data tree that represents the operational state is always a valid state 
data tree.


>
>     We have thought of several cases where it is possible that
>     duplicate entries could appear, and we don't want to force any
>     overhead on the device to guarantee that this does not occur, nor
>     do we want to force synchronization between configuration
>     processing and what is reported in <operational>.  Of course, in
>     normal circumstances this constraint, like the others, should not
>     be violated. This is already stated in the draft.
>
>     Examples of where uniqueness of list keys could be violated include:
>     1) If a device is internally paging the results back for a long
>     list, then it is possible that a entry could have been reported
>     near the beginning of the list, then moved, and then reported
>     again at the end of the list.
>
>
>
> The data is question is simply part of an <rpc-reply> and it is 
> subject to the validation
> requirements for RPC replies only.  Note also that the payload to 
> represent an arbitrary
> configuration datastore has to be done with anydata or anyxml.  RPC 
> reply validation
> rules for these nodes do not extend to contents of the anydata instance.

The text in section 4.7  is referring to the validity of the state data 
tree represented by <operational> rather than the contents of an 
<rpc-reply>.

The same considerations should also apply if the data is being streamed 
from the device via YANG push, or one of the other telemetry protocols.

>
>
>     2) If the list being stored somewhere in the system has become
>     corrupted and contains duplicate entries.  It is better to return
>     truth.
>
>
>
> It is better to fix the buggy server.
> Again, a representation of some portion of a datastore in an <rpc-reply>
> has nothing to do with the YANG validation of a datastore within a server.

I strongly agree that it is better to fix a buggy server, but it becomes 
much harder to fix if nobody is allowed to see that it is broken in the 
first place.  I.e. it is better to return the actual data that is wrong 
than to pretend that it is OK (e.g. by removing any entries that have 
duplicate keys before returning the data).

Do have any objections with the specific text change that I have 
proposed below.

Thanks,
Rob


>
>
>     3) On a distributed system where the list is being constructed
>     from multiple nodes (e.g. linecards or peer devices) then if a
>     list entry is moved from one node to another then it is again
>     possible that an entry could be reported in both places (or
>     neither) for a short interval before the system becomes consistent
>     again.
>
>
>
> Once again, this is an <rpc-reply> representation, not the validation 
> of a datastore
> within a server.
>
>
>
> Andy
>
>
>
>     Proposed text:
>
>     OLD:
>
>        <operational> SHOULD conform to any constraints specified in
>     the data
>        model, but given the principal aim of returning "in use" values, it
>        is possible that constraints MAY be violated under some
>        circumstances, e.g., an abnormal value is "in use", or due to
>     remnant
>        configuration (see Section 4.3.1).  Note, that deviations are still
>        used when it is known in advance that a device does not fully
>     conform
>        to the <operational> schema.
>
>        Only semantic constraints MAY be violated, these are the YANG
>     "when",
>        "must", "mandatory", "unique", "min-elements", and "max-elements"
>        statements.
>
>     NEW:
>
>        <operational> SHOULD conform to any constraints specified in
>     the data
>        model, but given the principal aim of returning "in use" values, it
>        is possible that constraints MAY be violated under some
>        circumstances, e.g., an abnormal value is "in use", the
>     structure of
>        a list is being modified, or due to remnant configuration (see
>        Section 4.3.1).  Note, that deviations are still used when it is
>        known in advance that a device does not fully conform to the
>        <operational> schema.
>
>        Only semantic constraints MAY be violated, these are the YANG
>     "when",
>        "must", "mandatory", "unique", "min-elements", and "max-elements"
>        statements; and the uniqueness of key values.
>
>     Again, if there are no objections, I will apply this change.
>
>     Thanks,
>     Rob
>
>
>     On 14/09/2017 16:44, Balazs Lengyel wrote:
>
>         See below !
>
>
>         On 2017-09-14 16:32, Martin Bjorklund wrote:
>
>                 CH 4.4.)  "Validation is performed on the contents of
>                 <intended>."
>                 This to me means that default data is not considered
>                 at validation
>
>             Note that RFC 7950, section 6.4.1, says:
>
>                 In the accessible tree, all leafs and leaf-lists with
>             default values
>                 in use exist (see Sections 7.6.1 and 7.7.2).
>
>             So defaults are taken into account when intended is validated.
>
>         BALAZS: Yes the two seem to contradict each other. This can be
>         understood in your way, however the current text is not clear
>         enough. I would add:
>         Validation is performed on the contents of <intended>
>         (EXTENDED WITH DEFAULT CONFIGURATION).
>
>                 which would be a backwards incompatible change. Also
>                 if validation
>                 does not consider system configured data that would
>                 allow cases like
>                 multiple interfaces named lo0. One from <intended> one
>                 from system
>                 configuration. IMHO while it is OK to violate
>                 uniqueness because of
>                 remnant data, the above violation of uniqueness seems
>                 a bad idea.
>
>             If your system adds data to <running>, or to <intended>,
>             it will be
>             validated.
>
>                 Ch. 4.7) Is it allowed to violate uniqueness of key
>                 values? IMHO it
>                 should not be.
>
>             Agreed.  Note that the draft explicitly lists the
>             constraints that can
>             be violated, and uniqueness of keys is not listed.
>
>         BALAZS: If that is the intent I would propose to explicitly
>         state it. For me it was non-trivial.
>         Can a a choice statement be violated? Having to existing
>         branches at the same time? It seems a semantic constraint to
>         me. IMHO yes.
>         Can an if-feature be violated? If  support has just changed
>         and we have some remnant config, I can very well imagine it
>         violated.
>
>         Also here could you change
>         If a node in  <operational> does not meet the syntactic
>         constraints then it cannot   be returned
>         to
>         If a node in  <operational> does not meet the syntactic
>         constraints then it MUST NOT be returned
>
>             /martin
>
>
>
>     _______________________________________________
>     netmod mailing list
>     [e-mail] <mailto:[e-mail]>
>     https://www.ietf.org/mailman/listinfo/netmod
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>