Re: [Netconf] FW: YP#12 Value filtering on non-key objects

[Andy Bierman <[e-mail]>]

On Wed, Mar 7, 2018 at 9:47 AM, Eric Voit (evoit) <[e-mail]> wrote:

> Robert and I chatted offline.  Here is our joint proposal for closing
> this.  (Alex confirmed he is ok too.)
>
> (1) Move "dampening" and "excluded change" so that they are covered by a
> single optional feature in the YANG Push draft.
>
> (2) A new trigger type which suppresses redundant changes / flaps without
> requiring notification could be covered in new drafts which extend YANG
> Push.  Such a trigger may or may not be built on RFC 8072 (YANG-Patch).
>
> (3) Metadata definitions which could encapsulate and summarize a set of
> changes since a previous notification could also be a new draft.
>
>
This sounds good.
I prefer that the draft is not specific about server implementation
details, such as specifying
when filter evaluation is done. It should be sufficient to say the content
delivered by the
server passes the filter.

I support the 3 drafts for publication.
It would be good for implementors to have a stationary target (for a change)


Eric
>
>
Andy


> > Hi Alex,
> >
> > Not sure what the "promise-theoretical" reasons are. ;-)
> >
> > I'm just checking that the solution is:
> >   - no more complex that it needs to be (at least for what I regard as
> > the core parts),
> >   - possible to implement efficiently in the publisher in a generic
> > way,
> >   - completely robust to failures.  In particular, the receiver
> > restarting, the publishing restarting, and transport issues.
> >
> > I don't see that publishing the same value as an indication of churn
> > meets all of the above goals.  I think that this this ends up
> > embedding potentially useful knowledge into an ephemeral message.
> > Other folks have presumably thought of different solutions, or perhaps
> > do not require the solution to be robust to all failure scenarios.
> >
> > I still don't get what a client does with a churn notification (in a
> > way that is robust over all failure scenarios) if it doesn't require
> > knowledge of the intermediate values.  And if it requires knowledge of
> > the intermediate values then per subscription dampening doesn't seem to
> really help.
> >
> > But I'm coming to the conclusion that my interpretation of why
> > "on-change" is useful is somewhat different to the one expressed in
> > rfc7923 and these drafts in WG LC.
> >
> > It seems that the purpose of on-change in rfc7923 is to potentially
> > allow a subscriber to know about all changes to a data node, along
> > with some dampening to reduce the number of updates.
> >
> > However, I see a different use of "on-change".  I regard YANG "on-change"
> > subscriptions as just a better mechanism to transfer operational state
> > data from a server to a client instead of polling with get requests.
> > There are two ways that I think it is better than GET requests, or
> polling:
> >
> >     (1) changes in the state are notified more quickly.  I.e. rather
> > than polling every 10/30/60 seconds, the data can be pushed
> > immediately when it is changed.
> >
> >    (2) Operational state data in steady state doesn't need to be
> > periodically transmitted reducing load on the server, client, and
> > bandwidth on the management network.
> >
> > In these scenarios, then for me, I think that the truly valuable
> information is:
> >   - what is the current value of a data node in the server (including
> > whether it no longer exists)
> >   - when did the server last update that value (e.g. to perform
> > reliable rate calculations on a counter).
> >
> > In terms of dampening, when I consider that parts of the network OS
> > that I'm most familiar with, I see that the dampening is generally
> > applied on a per object basis rather than a per subscription basis.
> > E.g. I don't want to wait 10 seconds to learn that my interface has
> > gone down because something else happened in my subscription, but
> > equally I don't want to see 1000 interface state changes reported per
> second.
> >
> > As such, I would prefer that for on-change subscriptions:
> >   - subscription level dampening is optional to implement.
> >   - "excluded-change" is optional to implement
> >   - for devices that support subscription level dampening, then a
> > subscriber chooses whether they require notification of no-op changes,
> > and it is optional to the publisher whether they implement this.
> >
> > I.e. make sure that basic stuff works well rather than requiring
> > everyone to implement bells and whistles that may in practice turn out
> > to be less useful than they are perceived to be.  I'll probably be
> > called in the rough on this issue.  Thankfully YANG has the deviation
> > statement ;-)
> >
> > Thanks,
> > Rob
> >
> > On 06/03/2018 19:49, Alexander Clemm wrote:
> > > Hi Rob,
> > >
> > > just to add my $0.02:
> > >
> > > I believe that the current solution is actually less expensive than
> > > some of the
> > things you are proposing, from maintaining counters to adding
> > information regarding timing of changes.  Some of the things you
> > suggest are useful but IMHO constitute additional extensions that can
> > be defined for periodic subscriptions.  On-change has its own set of
> > use cases and to have an indication that churn occurred during a
> > dampening period, even if it is later reverted,  is important to support
> for "promise-theoretical" reasons.
> > >
> > > --- Alex
> > >
> > >> -----Original Message-----
> > >> From: Netconf [mailto:[e-mail]] On Behalf Of Eric
> > >> Voit (evoit)
> > >> Sent: Monday, March 05, 2018 7:52 AM
> > >> To: Robert Wilton -X (rwilton - ENSOFT LIMITED at Cisco)
> > >> <[e-mail]>; [e-mail]
> > >> Subject: Re: [Netconf] YP#12 Value filtering on non-key objects
> > >>
> > >> Hi Rob,
> > >>
> > >>> Hi Eric,
> > >>>
> > >>>
> > >>> On 02/03/2018 22:47, Eric Voit (evoit) wrote:
> > >>>> Hi Rob,
> > >>>>
> > >>>>> Hi Eric,
> > >>>>>
> > >>>>>
> > >>>>> On 02/03/2018 17:22, Eric Voit (evoit) wrote:
> > >>>>>> Hi Rob,
> > >>>>>>
> > >>>>>>> Hi Eric,
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On 02/03/2018 14:44, Eric Voit (evoit) wrote:
> > >>>>>>>>> From: Eric Voit, December 22, 2017 2:52 PM
> > >>>>>>>>>
> > >>>>>>>>> I am looking to see if anyone has an issue with my opening
> > >>>>>>>>> and closing
> > >>> of:
> > >>>>>>>>> https://github.com/netconf-wg/yang-push/issues/12
> > >>>>>>>>> as described below...
> > >>>>>>>>>
> > >>>>>>>>> During Martin's recent review of yang-push, he argued that
> > >>>>>>>>> we should
> > >>>>> allow
> > >>>>>>>>> selection filtering using non-key properties.   I believe he
> is correct.
> > (In
> > >>>>> fact,
> > >>>>>>>>> this is what was supported in yang-push drafts -v00 through
> > >>>>>>>>> -v08.)
> > >>>>>>>>>
> > >>>>>>>>> The reason we changed in -v09 is that if a non-key property
> > >>>>>>>>> changes, it
> > >>>>>>> might
> > >>>>>>>>> mean that the object as a whole is no longer selected.    The
> result
> > is
> > >>> that
> > >>>>> an
> > >>>>>>>>> entire subtree may no longer should be visible to the receiver.
> > >>>>>>>>> So the YANG patch sent from the subscriber to the receiver
> > >>>>>>>>> would need to add/remove entire subtree based on a simple
> > >>>>>>>>> value change for a non-key
> > >>>>>>> object in that
> > >>>>>>>>> subtree.   The thought for -v09 was that if we restricted
> on-change
> > >>>>> selection
> > >>>>>>>>> filtering just to keys, this might be simpler/easier to
> implement.
> > >>>>>>>>> But per Martin's comments below, he pointed out other issues
> > >>>>>>>>> which come with placing such restrictions in filters.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Does anyone have an objections to once again allowing
> > >>>>>>>>> selection filters to
> > >>>>>>> be
> > >>>>>>>>> placed on non-key properties?    (Alex & Andy, this was
> something
> > we
> > >>>>> went
> > >>>>>>>>> back-and-forth on as part of the Dezign Team discussions.)
> > >>>>>>>>>
> > >>>>>>>>> If there is no objection, this makes the conceptual
> > >>>>>>>>> selection process for creating an push-change-update:
> > >>>>>>>> Above in this thread I proposed closing the issue YP#12
> > >>>>>>>> https://github.com/netconf-wg/yang-push/issues/12
> > >>>>>>>>
> > >>>>>>>> In a recent review, Tim Jenkins privately pointed out a bug
> > >>>>>>>> in the process
> > >>>>>>> below.   This bug has been fixed in the current -v15.
>  Basically the
> > process
> > >>>>> bug
> > >>>>>>> resulted from this thread last year when I forgot to document
> > >>>>>>> a step which
> > >>>>> has
> > >>>>>>> long been included in the draft text.    Specifically the text
> is from
> > Section
> > >>>>> 3.3
> > >>>>>>> On-change considerations: "If an object has returned to its
> > >>>>>>> original value (or even has been created and then deleted)
> > >>>>>>> during the dampening-period, the last change will still be sent.
> > >>>>>>> This will indicate churn is occurring on that object."
> > >>>>>>> I can't convince myself that this is really a robust way of
> > >>>>>>> indicating
> > >>>>>>> churn:
> > >>>>>>>
> > >>>>>>> - If an implementation retrieves an operational value via
> > >>>>>>> polling it may not know whether that value has changed between
> > polls.
> > >>>>>> Yes, this is a real issue of polling.
> > >>>>>>
> > >>>>>>> - If a client ends up loosing the connection, then it may miss
> > >>>>>>> out on notification that values have changed.
> > >>>>>> Yes.  If the connection is lost, a dynamic subscription ends,
> > >>>>>> so there is no
> > >>>>> guarantees of change indication.
> > >>>>>>> - If you have a chain of controllers, it seems to me more
> > >>>>>>> useful to try and suppress duplicate updates then propagating
> > >>>>>>> them through the
> > >>>>> network.
> > >>>>>> Duplicate updates are suppressed with the dampening interval.
> > >>>>>>
> > >>>>>>> Instead, if it is important to know that a particular value
> > >>>>>>> has changed then it may be better to have a separate leaf
> > >>>>>>> defining a count of the number of transitions that has
> > >>>>>>> occurred (e.g. the number of times an interface has flapped),
> > >>>>>>> or perhaps this could be defined more generically using a
> > >>>>>>> metadata annotation, or possibly a timestamp of when the value
> > >>>>>>> last changed (e.g. as is recommended in RESTCONF for
> > >>>>>>> configuration items).  Basically, any mechanism where the
> > >>>>>>> desired information is represented via explicit state rather
> > >>>>>>> than implicitly through
> > >>>>> the existence of the value in a notification.
> > >>>>>> This certainly is possible, and a quantity of flaps could be
> > >>>>>> augmented into the
> > >>>>> YANG patch definition for YANG-Push.  Or as you note, perhaps a
> > >>>>> timestamp with the latest flap could be placed into the YANG
> > >>>>> Patch definition.  However earlier discussions on flaps
> > >>>>> resulting in most just wanting a notification that a flap
> > >>>>> occurred.  Such a notification leaves open existing interfaces
> > >>>>> to directly query the logs for
> > >>> such counts.
> > >>>>>> I prefer the simpler definition.
> > >>>>> Sorry, but I don't see much real value in it:
> > >>>>> - clients cannot robustly infer anything from the duplicate
> > >>>>> value other that is the current value at the time of the
> notification.
> > >>>>> In fact, I think that the proposed text is actually somewhat
> > >>>>> unhelpful, since clients will probably think that they can infer
> > >>>>> something that they probably shouldn't if they want to be robust
> > >>>>> to failure
> > >> scenarios.
> > >>>> Consider the ACL security use case in slide 30:
> > >>>> https://datatracker.ietf.org/doc/slides-96-netconf-5/1/
> > >>>> what is needed is visibility into transient vulnerabilities.
> > >>>> However security
> > >>> solutions are not capable of ingesting undampened ACL changes in
> > >>> real
> > time.
> > >>> So dampening is needed without losing visibility of churn.
> > >>> Yes, no argument that the requirement is a valid one.
> > >> Excellent
> > >>
> > >>>   But I don't see how,
> > >>> once dampening is in place, that really solves the problem:  I
> > >>> would think that the only way a security solution would be able to
> > >>> determine that there was no vulnerable time interval would be by
> > >>> checking every ACL
> > >> update.
> > >>
> > >> Granular, targeted visibility of specific changes becomes available.
> > >> And then further analysis can be kicked off based on seeing a
> > >> dampened change.  I.e., if there is something unexpected seen,
> > >> targeted logs can be retrieved to get relevant ACL updates.
> > >>
> > >> I agree with you that if absolute, immediate validation is needed,
> > >> a solution should request a dampening period of zero.  And of
> > >> course there remains the option of pushing periodically if you are
> > >> not worried
> > about transient exposures.
> > >>
> > >> The ability to support this range of possibilities is a reason the
> > >> Security Automation and Continuous Monitoring WG added YANG Push
> > >> on-change to their charter in January:
> > >> https://datatracker.ietf.org/doc/charter-ietf-sacm/
> > >>
> > >>> My argument is whether the proposed solution is robust:  If you
> > >>> have an ACL update counter (acting as a logical timestamp) that is
> > >>> incremented whenever the ACL is changed then I think that there is
> > >>> a trivial solution to know whether an ACL has changed between two
> > >>> points in time (noting that it still doesn't tell you whether
> > >>> there was any window
> > >> where there is a potential vulnerability).
> > >>
> > >> There are different types of churning changes which can be made on
> > >> objects (add/delete nodes, changing values).  These churning
> > >> changes can be made up/down a tree.  Based on this, extending YANG
> > >> patch seems a reasonable approach to identify churning changes made
> > >> since the
> > last update.
> > >>
> > >> I don't dispute that a robust mechanism which augments YANG push to
> > >> count the number of dampened changes might be constructed.  I would
> > >> also think that any such a mechanism would *also* naturally be used
> > >> to
> > count all the changes made,
> > >> even if the changes don't cancel each other out.  Agree?   If yes, it
> looks like
> > we
> > >> are looking for a different and robust type of summary information
> > >> coming
> > out
> > >> of the patch operation.  And this is not a bad thing.   (Perhaps
> someone
> > wants to
> > >> propose a specific algorithm which can augment the YANG patch
> > >> operation?)
> > >>
> > >>> I guess that the crux of my concern is that I perceive that it
> > >>> natural for systems to want to suppress "no change" events rather
> > >>> than flooding them on to other systems, e.g. to handle device
> > >>> restarts where it is likely that the vast majority of notified
> > >>> data may be unchanged.  But here you have notifications that are
> > >>> expected to always flood onward, because it is the notification
> > >>> message itself that has semantic meaning beyond the change in
> > >>> value that is being
> > notified.
> > >> Yes this makes sense.   But if a publisher makes a promise to a
> subscriber to
> > >> inform them of changes to the datastore, then this is what it needs
> to do.
> > >>
> > >> This is one of the reasons the RFC7923 requirements say:
> > >>
> > >>       A Subscription Service SHOULD support the ability to subscribe
> to
> > >>       updates on-change, i.e., whenever values of subscribed data
> objects
> > >>       change.
> > >>
> > >>       For on-change updates, the Subscription Service MUST support a
> > >>       dampening period that needs to be passed before the first or
> > >>       subsequent on-change updates are sent.  The dampening period
> > SHOULD
> > >>       be configurable as part of the subscription request.
> > >>
> > >> A publisher may certainly expose alternative/new types of
> > >> subscription if it is unable to meet these bars.
> > >>
> > >>>    Further, I think that generating those notifications robustly
> > >>> is going  to require a complex implementation.
> > >> This can be true.  Different components have various levels of
> > >> challenge in how to support.  This is why deployable solutions will
> > >> need ways of indicating which objects support on-change.
> > >>
> > >>>> If a vulnerability is found off-box, more details can be pulled
> > >>>> from existing
> > >>> logs using existing operations.  This provides a minimal, clean
> solution.
> > >>> For the security solution to be robust it must surely examine
> > >>> every change to the ACL.  I can't see how dampening can avoid that
> > >>> requirement in any sort of robust way.
> > >> It doesn't avoid it.   It just gives an indication to the application
> where more
> > >> analysis might be needed.  It is up to the application to determine
> > >> things which can be safely ignored, and/or make correlations to
> > >> known configuration operations underway.
> > >>
> > >>>>
> > >>>> To build a more robust solution, we would need define YANG
> > >>>> structures
> > >>> which are capable of summarizing the times and timeframes of
> > >>> different types of churning data.  (IPFIX does stuff like this
> > >>> with inter-packet-gap histograms, so it is possible.)  We would
> > >>> then need to place those structures into the YANG patch.  This
> > >>> would be non-trivial.  I would suggest we get some feedback from
> > >>> the existing draft
> > >> before extending to do more here.
> > >>> I'm not suggest doing more now.  I'm suggesting that it could be
> > >>> simplified or made optional.
> > >> As per above, RFC 7923 defines on-change as "whenever values of
> > subscribed
> > >> data objects change".   And dampening support is mandatory where on-
> > change
> > >> is supported.
> > >>
> > >>> It seems to me that this requirement adds a lot of complexity to
> > >>> the implementation.  Consider a list with entries that has been
> > >>> added/removed.  Presumably the implementation needs to track all
> > >>> of these transient list entries after they have gone away, to
> > >>> generate a delete notification for something that isn't there.
> > >>> Similarly for containers that have been created and then deleted
> > >>> before a notification is due to be generated.  Perhaps there is a
> > >>> easy solution that I'm missing, but thinking about this, I think
> > >>> that this is going to be
> > >> complex/costly to implement.
> > >>
> > >> Yes, sometimes it will be expensive.   But the alternative is having
> a promise
> > >> which isn't reliable/meaningful to the subscriber.  I.e., without
> > >> such a
> > guarantee
> > >> it places into question the usefulness of the provided data.   And we
> end up
> > >> having just another form of periodic reporting.
> > >>
> > >>>>> - servers may not be able to generate it reliably, or it may be
> > >>>>> expensive/complicated to do it.  It is no longer enough just to
> > >>>>> compare old and new values, instead servers will need to always
> > >>>>> maintain additional state to try and generate this notification.
> > >>>> What differentiates on-change from periodic is the guarantee to
> > >>>> the
> > >>> subscriber that it gets to know if a change occurred.  An
> > >>> on-change which doesn't actually expose changes is not something we
> should allow.
> > >>> Why not?
> > >>>
> > >>> We have exactly this sort of system for suppressing carrier level
> > >>> link flaps without propagating the flap to higher layers.  If a
> > >>> flap is short (e.g. perhaps 10
> > >>> msecs) then the link flap is suppressed entirely, and is not seen
> > >>> by the rest of the system.
> > >> Yes, those internally dampened changes are not propagated to the YANG
> > >> datastore.   And that is what the boundary of measurement is here:
> can you
> > see
> > >> a change in the YANG datastore.
> > >>
> > >> What MUST be avoided is the possibility that someone did a GET to
> > >> the YANG datastore, and found a value different than seen by a
> > >> subscriber.  And that subscriber never was given any indication of
> churn.
> > >>
> > >>> Instead we have a separate counter that reports the number of
> > >>> actual carrier transitions that occur.  Clients can monitor the
> > >>> carrier transition counter to determine whether flaps in carrier
> > >>> signals are
> > occurring.
> > >> That counter may or may not be on-change subscribable.   It is great
> that
> > some
> > >> systems have this.
> > >>
> > >>> Another example, would be a operator monitoring a network.  I've
> > >>> heard that some ISPs don't page their ops engineers if a link
> > >>> flaps once for a short period of time and then automatically
> recovers.
> > >>> They care a lot more if it is happening repeated, or if it doesn't
> recover.
> > >> Such an application is easily built on dampened changes *if* the
> > >> changes
> > are
> > >> visible in the YANG datastore.   See slide 29 of:
> > >> https://datatracker.ietf.org/doc/slides-96-netconf-5/1/
> > >>
> > >> Such an implementation would be very difficult if (a) dampening
> > >> were not available, or suppressed, or (b) a solution doesn't have
> > >> the transitions counter you mention above.
> > >>
> > >>> Both of these systems are operating a dampened on change
> > >>> notification, but where the notifications are entirely suppressed
> > >>> if the system has reverted back to their original state.
> > >> Both seem quite doable with the current on-change YANG patch
> definition.
> > >>
> > >> Thanks,
> > >> Eric
> > >>
> > >>> Thanks,
> > >>> Rob
> > >>>
> > >>>
> > >>>> (Of course it is ok to reject a subscription for an object which
> > >>>> can't support
> > >>> dampened on-change for reasons described above.   And periodic is
> always
> > an
> > >>> option.  Another option might be that a publisher suggesting a
> > >>> dampening period hint of zero.)
> > >>>>> I would actually propose that you write something along the
> > >>>>> opposite
> > lines:
> > >>>>> - servers may generate duplicate notifications even though the
> > >>>>> value hasn't changed (e.g. due to the state flapping).
> > >>>>> - clients cannot reliably infer anything from a duplicate value.
> > >>>> I think you are saying On-change is accomplished via a YANG patch
> > >> operation.
> > >>> I am not sure how duplicates patches would play out here.
> > >>>>> Alternatively, on the presumption that you don't want to change
> > >>>>> this, then would it make sense to make this optional/conditional
> > >>>>> in some
> > >> way?
> > >>>> Optional makes it very close to the periodic behavior.   An
> on-change
> > needs
> > >>> to indicate when some change has occurred.
> > >>>> As an FYI, for some earlier conversations in this space, check
> > >>>> the "Periodic
> > >>> Eventing" options within:
> > >>>> https://www.ietf.org/mail-archive/web/netconf/current/msg12375.ht
> > >>>> ml There are lots of ways this work could be extended.
> > >>>>
> > >>>> Thanks
> > >>>> Eric
> > >>>>
> > >>>>
> > >>>>> Thanks,
> > >>>>> Rob
> > >>>>>
> > >>>>>
> > >>>>>> Eric
> > >>>>>>
> > >>>>>>> Thanks,
> > >>>>>>> Rob
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> To fix this bug the following text has been inserted as an
> > >>>>>>>> independent step
> > >>>>>>> (4)...
> > >>>>>>>>>      (1) Just before a change, or at the start of a
> > >>>>>>>>> dampening period, evaluate
> > >>>>>>> any
> > >>>>>>>>> filtering and any access control rules.  The result is a set
> "A"
> > >>>>>>>>> of datastore
> > >>>>>>> nodes
> > >>>>>>>>> and subtrees.
> > >>>>>>>>>
> > >>>>>>>>>      (2) Just after a change, or at the end of a dampening
> > >>>>>>>>> period, evaluate any filtering and any (possibly new) access
> > >>>>>>>>> control rules.  The
> > >>>>> result is a set "B"
> > >>>>>>> of
> > >>>>>>>>> datastore nodes and subtrees.
> > >>>>>>>>>
> > >>>>>>>>>      (3) Construct a YANG patch record for going from A to B.
> > >>>>>>>>      (4) If there were multiple changes made to a subscribed
> > >>>>>>>> object between
> > >>>>> "A"
> > >>>>>>> and "B" which fully canceled each other out, then if access
> > >>>>>>> control permits read access, insert into the YANG patch record
> > >>>>>>> the last change
> > >>>>> made to that object.
> > >>>>>>>>>      (5) If the resulting patch record is non-empty, send it
> > >>>>>>>>> to the
> > receiver.
> > >>>>>>>> Final note, while writing this email update, I did a text
> > >>>>>>>> tweak from step (4)
> > >>>>>>> from what is in -v15.  But the intent of the text is the same.
> > >>>>>>> And the process now returns to matching both earlier versions
> > >>>>>>> of the draft, and historical documentation provided such as
> > >>>>>>> slides
> > >>>>>>> 29 & 30 from
> > >>>>> IETF96.
> > >>>>>>>> https://datatracker.ietf.org/doc/slides-96-netconf-5/1/
> > >>>>>>>>
> > >>>>>>>> Eric
> > >>>>>>>>
> > >>>>>>>>> Eric
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>> From: Martin Bjorklund,  December 5, 2017 5:03 AM
> > >>>>>>>>>>
> > >>>>>>>>>>>>>>          the goal is to
> > >>>>>>>>>>>>>>          provide equivalent capabilities to what is
> > >>>>>>>>>>>>>> available with a
> > >>> GET.
> > >>>>>>>>>>>>>>       In GET you can filter on "non-key properties".
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>       I think you should remove the text about "non-key
> > >>> properties".
> > >>>>> If
> > >>>>>>>>>>>>>>       anything, I think you can allow an implementation
> > >>>>>>>>>>>>>> to reject a
> > >>>>> filter
> > >>>>>>>>>>>>>>       that would be too complex to implement / evaluate
> > >>>>>>>>>>>>>> (in fact I
> > >>>>> think
> > >>>>>>>>>>>>>>       the text already allows a server to reject such
> > >>>>>>>>>>>>>> filters.)
> > >>>>>>>>>>>>> The issue with non-key properties is how to deal with
> > >>>>>>>>>>>>> the creation and deletion of objects when a non-key
> > >>>>>>>>>>>>> property goes in/out of the selection.  Others were
> > >>>>>>>>>>>>> uncomfortable with things like passing along a patch
> > >>>>>>>>>>>>> showing a node was deleted, when in reality the property
> > >>>>>>>>>>>>> simply changed from meeting the selection filter to
> > >>>>>>>>>>>>> failing the selection filter.  So people wanted to
> > >>>>>>>>>>>>> default to the simpler case of key properties for
> > >>> selection.
> > >>>>>>>>>>>> But you are constraining the entire solution to support
> > >>>>>>>>>>>> just this use case, when a simpler unconstrained solution
> > >>>>>>>>>>>> just as easily supports this use case, *plus other use
> > >>>>>>>>>>>> cases* - if a client does not want these "fake *deletes",
> > >>>>>>>>>>>> they can simply construct filters based on keys only,
> > >>>>>>>>>>>> which they would in the current
> > >>>>> solution anyway.
> > >>>>>>>>>>> This is true.
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Note also that b/c of the last paragraph in section 3.9,
> > >>>>>>>>>>>> a client must deal with such deletes anyway, even if it
> > >>>>>>>>>>>> specificied only keys.
> > >>>>>>>>>>> This is true.
> > >>>>>>>>>>>
> > >>>>>>>>>>>> But I do understand your point.  This is also related to
> > >>>>>>>>>>>> my comment below about when a filter is evaluated.  Based
> > >>>>>>>>>>>> on you reply to that comment, I think the conceptual
> > >>>>>>>>>>>> algorithm is like
> > >> this:
> > >>>>>>>>>>>>       Just before a change, evaluate the filter and any
> > >>>>>>>>>>>> access
> > control
> > >>>>>>>>>>>>       rules.  The result is a set "A" of nodes (including
> subnodes).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>       Just after a change, evaluate the filter and any
> > >>>>>>>>>>>> (possibly
> > new)
> > >>>>>>>>>>>>       access control rules.  The result is a set "B" of
> > >>>>>>>>>>>> nodes
> > (including
> > >>>>>>>>>>>>       subnodes).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>       Construct a YANG patch record for going from A to B.
>  If the
> > >>> record
> > >>>>>>>>>>>>       is non-empty, send it to the subscriber.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> (this is conceptual, an implementation can do lots of
> > >>>>>>>>>>>> optimizations to avoid evaluating filters on un-related
> > >>>>>>>>>>>> changes)
> > >>>>>>>>>>> Yes, this is it.  The only tweak I would make is to add
> > >>>>>>>>>>> the dampening period.  As a result, I have added the
> > >>>>>>>>>>> following text at the end of the "On-Change
> > >>>>>>>>>>> Considerations" section
> > >>>>>>>>>>>
> > >>>>>>>>>>> "Putting it all together, following is the conceptual
> > >>>>>>>>>>> process for creating an push-change-update notification:
> > >>>>>>>>>>>
> > >>>>>>>>>>>       1.Just before a change, or at the start of a dampening
> period,
> > >>>>>>>>>>>       evaluate any filtering and any access control rules.
> > >>>>>>>>>>> The result
> > is a
> > >>>>>>>>>>>       set "A" of datastore nodes and subtrees.
> > >>>>>>>>>>>
> > >>>>>>>>>>>       2.Just after a change, or at the end of a dampening
> > >>>>>>>>>>> period,
> > >>> evaluate
> > >>>>>>>>>>>       any filtering and any (possibly new) access control
> > >>>>>>>>>>> rules.  The
> > >>> result
> > >>>>>>>>>>>       is a set "B" of datastore nodes and subtrees.
> > >>>>>>>>>>>
> > >>>>>>>>>>>       3. Construct a YANG patch record for going from A to B.
> > >>>>>>>>>>> If the
> > >>> record
> > >>>>>>>>>>>       is non-empty, send it to the receiver."
> > >>>>>>>>>>>
> > >>>>>>>>>>>>> Based on that I can delete the words "but the goal is to
> > >>>>>>>>>>>>> provide equivalent capabilities to what is available
> > >>>>>>>>>>>>> with a
> > GET".
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>       With the current text, is this filter ok:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>          /interfaces/interface[contains(name, "eth")]
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>       It filters on a key, so it should be ok, right?
> > >>>>>>>>>>>>> Yes
> > >>>>>>>>>>>> What about this one:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>        /interfaces/interface[contains(name, //name)]
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> it also filters on a key is it ok?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> [if the answer is "yes" we have a problem; and if it is
> > >>>>>>>>>>>> "no", we need to tighten the text]
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> All this makes me wonder if it is correct to have "subtree"
> > >>>>>>>>>>>> and "xpath"
> > >>>>>>>>>>>> filters at all.  If they only can match on keys, they
> > >>>>>>>>>>>> become severely constrained.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> An alternative could be to specify filters as
> > >>>>>>>>>>>> "nacm:node-instance-identifier"
> > >>>>>>>>>>>> instead; these are instance-identitifiers that allow
> missing keys.
> > >>>>>>>>>>>> For
> > >>>>>>>>>>>> example:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>       /if:interfaces/if:interface/if:oper-status
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This would be easier to understand and probably easier to
> > >>>>>>>>>>>> optimize for in the server code.
> > >>>>>>>>>>> I want to generalize this "only-keys?" issue for the full WG.
> > >>>>>>>>>>> It is too important to handle this deep in the thread.
> > >>>>>>>>>>> Look for me to open an issue shortly.
> > >>>>>>>>>> Ok!
> > >>>>>>>>>>
> > >>>>>>>>>> /martin
> > >>>>>>>> _______________________________________________
> > >>>>>>>> Netconf mailing list
> > >>>>>>>> [e-mail]
> > >>>>>>>> https://www.ietf.org/mailman/listinfo/netconf
> > >>>>>>>> .
> > >>>>>>>>
> > >> _______________________________________________
> > >> Netconf mailing list
> > >> [e-mail]
> > >> https://www.ietf.org/mailman/listinfo/netconf
>
> _______________________________________________
> Netconf mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netconf
>