Re: [Netconf] Subscribed-notifications, replay objects

["Eric Voit (evoit)" <[e-mail]>]

Hi Balazs,

There is value of being able to resend/replay lost on-change updates.  Let's figure out the best way to frame this.

First let's establish the scope.  As datastore changes need to be applied in sequence, patching a datastore with replayed data could require that you first back out any subsequent changes which were applied on the receiver's objects (if any).  Is Ericsson considering the backed-out scenario as part of replay scope?   At this point I am assuming no, but we don't want to define any technology mechanisms which might preclude this in the future.

Per the discussion during today's NETCONF session.  It seems risky to ask for on-change replay based on timestamps (at least for generalized implementations).  There are two issues for me here:

(1) Which timestamps should be used, & what about notification delay? A pure time based replay as part of a generic on-change capability could have loss/duplication, particularly as change-updates can be dampened and batched.

*       As you noted in your parallel thread: "Many systems don't differentiate between notification time and record-time."

*       With dampening, you need to know when the update actually got sent.  I.e., you need to look at the notification time rather than the time a change was made to a datastore.

*       When replay state time & notification time timestamps are really close, maintaining the integrity or the receiver's datastore extract would need to be done by looking at whether a notification was sent or not.

(2) Replay from what?

*       An attempt to replay on-change notifications means that there was a previous dynamic subscription with identical parameters previously in place.  In addition, it must be possible to reconstruct deltas from that previous dynamic subscription. (This is not an issue for configured subscriptions, and therefore it is not an issue for the multiple-receiver case.)

*       There is not a meaningful difference in providing one patch operation representing all changes, and incremental patch operations.

Based on this, it would seem to me that on-change replay should be done based on an augmentation to the establish-subscription RPC.  This augmentation would allow an the addition of the input object "previous-subscriptions-last-notification-id".  Having that parameter tightly focuses the viable use and security of this type of replay.  As notification-id is defined in draft-voit-netconf-notification-messages, it seems like we need this nail this down in conjunction with that draft.  Perhaps the RPC augmentation could be added here?

BTW: In the original version of draft-voit-netmod-yang-notifications2, we allowed random access in case receivers wanted to get past information.  This random access was seen as overkill by some.  If we want a more robust replay capability than above, we should revisit this.

Eric

From: Balazs Lengyel [mailto:[e-mail]]
Sent: Thursday, July 20, 2017 9:40 AM
To: Eric Voit (evoit) <[e-mail]>; [e-mail]
Subject: Re: [Netconf] Subscribed-notifications, replay objects


Hello Eric,

We have been using replay for on-change notifications in our own ericsson solution. I would not like to loose that possibility. I agree that replay is not that useful on periodic subscriptions, but it is usable for on-change. We use on-change notifications for configuration mirroring. If we loose the notification session for some time, it is much cheaper to replay the lost notifications then to get a full config update.

regards Balazs

On 2017-07-11 23:55, Eric Voit (evoit) wrote:
In an email earlier today, Martin pointed out replay was under-defined.  To fix this, below is suggested text and yang objects.  The goal of this is keep this as close to RFC-5277 as possible.


So following would replace the text in Section 4.1.1



Replay provides the ability to establish an event subscription which is also capable of passing along recently generated events.  In other words, as the subscription initializes itself, it sends any previously generated notifications for the target event stream which meet the filter and timeframe criteria.  These historical events would then be followed by events generated after the subscription has been established.  All events will be delivered in the order generated.  Replay is only viable for dynamic subscriptions.  Replay is an optional feature.   Replay is dependent on a notification stream supporting some form of notification logging, although it puts no restrictions on the size or form of the log, or where it resides within the device.



The presence of a replay-start-time within an establish-subscription RPC is the way a replay may be been requested.  The provided start time must be earlier than the current time.  If the start time points earlier than the maintained history of Publisher's event buffer, then the subscription must be rejected.  In this case the error response to the <establish-subscription> request should include a start time supportable by the Publisher. An end time may be specified using the optional stop-time parameter, which may also be earlier than the current time.  If no stop-time is present, notifications will continue to be sent until the subscription is terminated.



Not all streams will support replay.  Those that do must include the <replay-supported> flag.  In addition, a notification stream that does support replay is not expected to have an unlimited supply of saved notifications available to accommodate any given replay request.  Subscribers may do a get on <replay-log-creation-time> and <replay-log-aged-time> to assess the availability of notifications for replay.  The actual number of stored notifications available for retrieval at any given time is a publisher specific matter.   Control parameters for this aspect of the feature are outside the scope of this document.





The updated data model tree in Section 3 will includes:.
    +--ro streams
       +--ro stream* [stream]
          +--ro stream
          +--ro description
          +--ro replay-support?
          +--ro replay-log-creation-time?
          +--ro replay-log-aged-time?


And for the YANG model in Section 9, we have:
(Note: all definitions taken from RFC-5277 section 3.4)

  container streams {
    config false;
    description
      "This container contains information on the built-in streams
      provided by the publisher.";
    list stream {
      key "stream";
      description
        "Identifies the built-in streams that are supported by the
         publisher.";
      leaf stream {
        type stream;
        description
          "A handle for a sequential set of events, each of which
           is characterized by its own domain and semantics.
           In case configurable custom streams are supported,
           as indicated by the custom-stream identity, the configuration
           of those custom streams is provided separately.";
      }
      leaf description {
        type string;
        mandatory true;
        description
          "A description of the event stream, including such information
           as the type of events that are sent over this stream.";
      }
      leaf replay-support {
        type empty;
        description
          "Indicates that event replay is available on this stream.";
      }
      leaf replay-log-creation-time {
        type yang:date-and-time;
        description
          "The timestamp of the creation of the log used to support the
          replay function on this stream. Note that this might be
          earlier then the earliest available notification in the log.
          This object is updated if the log resets for some reason.
          This object MUST be present if replay is supported.";
      }
      leaf replay-log-aged-time {
        type yang:date-and-time;
        description
          "The timestamp of the last notification aged out of the log.
          This object MUST be present if replay is supported and any
          notifications have been aged out of the log.";
      }
    }
  }

As always, thoughts and comments welcome.

Eric




_______________________________________________

Netconf mailing list

[e-mail]<mailto:[e-mail]>

https://www.ietf.org/mailman/listinfo/netconf



--

Balazs Lengyel                       Ericsson Hungary Ltd.

Senior Specialist

Mobile: [phone]              email: [e-mail]<mailto:[e-mail]>