Re: [YANG] key clause issues

Martin Bjorklund wrote:
> Andy Bierman <ietf@andybierman.com> wrote:
>> Martin Bjorklund wrote:
>>> Hi Andy,
>>>
>>> Andy Bierman <ietf@andybierman.com> wrote:
>>>> Issue 1)
>>>>
>>>>    From an interoperability perspective, the key needs to
>>>>    be defined any time there are multiple named instances
>>>>    of an element (i.e., every list).
>>> Exactly what will not be interoperable by not having keys for
>>> non-config data?
>>>
>> The feature intent needs to be explained in the draft.
>> If a list has no key defined, then that means that
>> there is no distinction in the agent between any of
>> the entries, other than their relative position (foo[1], foo[2], etc.)
>>
>> If this is true, and the manager knows that, then there is no
>> problem leaving out the key.  (Other DMLs call this multiple
>> instances of a container, not a list though).
>>
>> But if the agent has some way of identifying list entries,
>> other than their relative position, then a key MUST be specified.
>> (Is this what the draft intends?)
> 
> I dont't follow.  How does it matter to interoperability if the agent
> has some way of identifying list entries?  Which protocol operation
> will suffer?
> 

See below

> 
>>> (One problem might be how to define access control.  W/o keys you
>>> might have to fall back to generic XPath, which we want to avoid...)
>>>
>>>>    Config vs. read-only is irrelevant.
>>>>    Operators and manager applications need to know how list entries
>>>>    are indexed for interoperability.  SNMP is mostly for monitoring.
>>>>    I can't imagine how operators and DM readers would be able to talk about
>>>>    tables and rows if all the INDEX clauses were removed from read-only tables.
>>> In SNMP, the INDEX is needed by the protocol.  Not so in NETCONF for
>>> non-config data.
>> Not true at all.  If the agent is using named entries
>> instead of position-identified entries, the manager needs to know.
>>
>> Consider any XML table like the ifTable counters.
>> What if the agent has entries for interfaces 1, 2, and 3,
>> but after awhile, interface 2 is deleted.  If the agent
>> knows about 'ifIndex', but not the manager, how well do
>> you think the manager application will handle this common scenario?
> 
> ifIndex is a leaf that the manager knows about, and so does the
> agent.  What will not work well?

Yes, the manager knows about all the leafs (ifIndex, ifInOctets, ifMtu, etc.)
Without the magic text 'key ifIndex;', the manager has no idea what
makes one blob called <ifEntry> from another one.

Without a key, the manager has to identify each list node
by its position:

   <ifEntry>
     <ifIndex>1</ifIndex>
     <ifMtu>1500</ifMtu>
     ...
   </ifEntry>
   <ifEntry>
     <ifIndex>2</ifIndex>
     <ifMtu>9600</ifMtu>
     ...
   </ifEntry>
   <ifEntry>
     <ifIndex>3</ifIndex>
     <ifMtu>9600</ifMtu>
     ...
   </ifEntry>

Now, if the manager application has no idea that 'ifIndex' is the
key, and the entire <ifEntry> for entry '2' is deleted,
then the manager has no idea it is collecting data
for entries 1 and 3, not 1 and 2.

> 
> [I am playing the devil's advocate here.  I would be perfectly happy
> with keys for non-config data as well.]
> 
> Ah, now I remember the other reason.  If I use a list in an rpc or
> notification, do I still need to have a key?

yes -- if there are named entries in the list
no -- if the list has unnamed entries, just position tells them apart

>   
>> Config vs. non-config is irrelevant here.
>> Identification via named keys or identification via node position
>> is the issue.
>>
>>
>>>
>>>> Issue 2)
>>>>
>>>>    Why is the CLR about a leaf appearing more than once in the key needed?
>>>>    It should be up to the DM writer to decide if this makes sense or not:
>>>>
>>>>     list addr-pairs {
>>>>       key "addr-type src-addr addr-type dest-addr";
>>> What does this mean?  What is the difference between this and:
>>>
>>>         key "addr-type src-addr dest-addr";
>>>
>>> What does an instance identifier for your example look like?
>>>
>> This table obviously cannot use the nested address.
>> By design, it uses separated fields.
> 
> You lost me again.  What does nested address have to do with the
> single-occurance of leaf in the key?
> 
> I still don't understand what the key above actually is supposed to
> mean.

Instance ID:

   /addr-pairs[addr-type='ipv4'][src-addr='192.168.0.1'][dest-addr='192.168.0.4']

> 
>>>> Issue 3)
>>>>    Why is the key statement limited to top-level sibling nodes
>>>>    within the list?
> 
> [...]
> 
>>> Also, by allowing deep keys, you force an agent implementation to do
>>> buffering of - in the worst case - the entire configuration.  You
>>> influence the protocol characteristics to the worse.  W/o deep keys
>>> implementations _can_ do minimal buffering, and thus make the NETCONF
>>> protocol usable in smaller devices.  I know at least three
>>> implementations that work this way today (and I know one that does
>>> buffering).
>>>
>> This is really irrelevant in the context of standardization.
> 
> Is it?  Even XSD puts restrictions on the XPath used in keys "In order
> to reduce the burden on implementers, in particular implementers of
> streaming processors"
> [http://www.w3.org/TR/xmlschema-1/#cIdentity-constraint_Definitions].
> 
> Wasn't one of the reasons for not doing XPath filtering only in
> NETCONF that it would require more resources on the device?
> 
> If we get a YANG WG, I think it is very important the WG really
> understands the implementation/resource implications of constructs
> like this.
> 

I do not think buffering input is an implementation burden
that should be a big factor.  The size of the data models
on the device is going to be relative to the device.

WGs and vendors should certainly attempt to place the index leafs
first instead of last, but should not be forced to do so.

Most importantly though,  WGs and vendors should not be forced
to undo (cut-and-paste) data placed in containers, just so
a list can use that data in the key.

It must be up to the DM writer where to use containers
for data, instead of spreading everything out in a flat table
like SMIv2.  Simple containers like (type, addr, port)
are going to be reused, and going to be needed in list keys.

SMIv2 never had any nesting.
That is a bug, not a feature.
XML DM writers need to be able to use nesting anywhere
it makes sense for a particular data model.

>> In other implementations (like mine) the position of the key
>> within the list element sub-tree is not a factor.
>>
>> BTW, there is no text in the draft that says that the key components
>> must be defined first in the list.  In order to optimize the DML
>> design in favor of this implementation choice, the key components
>> can never be in reusable groupings (like address/port) and they
>> must always be the first leafs, defined in the same order as the key.
> 
> The keys don't have to be defined first.  They will be encoded first
> in the XML.
> 
>> SMIv2 never had any massive CLRs like this for tables.
> 
> Well, in SMI there were no deep structures.
> 
>> You can specify 'key' clauses in XSD for arbitrary nodes.
> 
> Right.  Are you arguing that we should support the same flexibility of
> keys as XSD?

no.  XSD allows the key to be outside the list.
In YANG, the key components MUST be defined within the list,
either directly, or via expansion of 'uses' statements.

I am proposing:
   1) The key leafs do no have to be first, or in any particular
      order within the list
   2) leafs nested only within the 'container' construct may
      be used as key components

> 
>> There is so much complexity in YANG wrt/ Xpath, local typedefs/groupings,
>> augments, that this hardly seems like an implementation burden
>> worthy of such drastic restrictions on data model design.
> 
> First of all, YANG does not require an XPath engine on the device, for
> the same reason - to reduce the implementation burden on the device.
> 
> Local typedefs / groupings might add complexity to the compiler, but
> not to the engine.
> 
> And again, it is the resource consumption on the device that I'm
> worried about.  By allowing this you essentially say that streaming
> processing is out of the question.
> 
> 
>>>>    With this constraint on the key-stmt, YANG cannot support
>>>>    indexing using reusable containers.  (There needs to be
>>>>    exactly one instance of the key component per list entry,
>>>>    so 'list' and 'choice' nodes cannot be used within a key.)
>>> I can also argue that this is an CLR.  Why don't you allow:
>>>
>>>    list foo {
>>>       key address;
>>>       container address {
>>>          choice type {
>>>             leaf ipv4address;
>>>             leaf ipv6address;
>>>             leaf name;
>>>          };
>>>          leaf port;
>>>       }
>>>    }
>>>
>>
>> We need to understand what a key is, and when it is used.
>> A key identifies the static naming components so multiple
>> instances of a list node can be distinguished by an application.
>>
>> I am not proposing that the key components be complex and/or non-static,
>> like you are, with this straw-man example.   It is not a CLR to
>> specify that key components must be static leafs.  This is
>> how SMIv2 works, and it should be how YANG works as well.
> 
> Again, SMI does not have deep structures in tables so I don't think
> the comparison helps.
> 
> 
> Out of curiosity - how do you handle deep keys in your CLI?

Yet another implementation detail, but the CLI processing
is different than NETCONF.  For CLI, all the index components
are listed first, like:

  ifEntry 1 {
    # ifIndex would not be listed since it is the index
    ifMtu 1500
    ifName Ethernet0/0
  }

Since the 'ifIndex' node has a back-ptr to its index header,
it can be skipped correctly no matter where it is located
in the subtree.

> 
> 
> /martin
> 
> 

Andy

_______________________________________________
YANG mailing list
YANG@ietf.org
https://www1.ietf.org/mailman/listinfo/yang