Re: [OPSAWG] Manifest need? Re: draft-claise-opsawg-collected-data-manifest

[Benoit Claise <[e-mail]>]

Hi Alex,

Sorry for the delay in replying.

You are right in the sense we don't define new piece of information. I 
stressed it on the first slide during the IETF 115

    •Goal is not to expose new information via YANG but rather to define
    what needs to be kept as metadata (or Data Manifest) to ensure that
    the data can still be interpreted correctly even:
    –if the source device is not accessible (from the collection system)
    –If the source device has been updated or has a new configuration


Let me make a distinction between two different use cases.
1. configuration management. The client knows every configuration 
parameters. No problem here
2. monitoring: device monitoring onboarding and network analytics

In case 2 (the focus of our draft), typically, new devices are installed 
in the network, and the analytics must be automatically consumed 
(assuming the telemetry is already configured) throughout the monitoring 
architecture:
     1. YANG Push publisher (=router)
     2. YANG Push receiver (=collector)
     3. Message broker
     4. TSDB
     5. Analytic tool
Think of the IPFIX analogy (for which its onboarding is done 
automatically, as you send the template along with the data record).

Back to YANG push.
Yes, the telemetry receiver could query YANG models with information 
such as RFC8641 YANG module, YANG-library, software and hardware info, 
etc), and forward the information.
And yes, we could subscribe to YANG push subscription configuration 
data, or potentially other data.

What we want to achieve here is the ability for context information to 
follow the data, from 1 to 5 above, without explicit subscription (as 
Tianrian mentioned), in a standardized way.

Yes, we will clarify this in the draft. Thanks for your feedback.

Regards, Benoit

On 11/18/2022 5:39 PM, Alexander Clemm wrote:
>
> So, saving the invocation of one command to establish the 
> “meta-subscription”, that’s all this is for?  And for this we are 
> asking implementors to create what is in effect redundant 
> instrumentation?  This seems a lot of effort for a small saving.
>
> As a thought, even in that case, would you even need a redundant 
> YANG-data model, or would it make sense to instead augment the 
> existing model with an additional parameter instead “provide manifest 
> info”, which adds implicitly a subscription to the subscription info 
> to the subscription itself?  Such an alternative could be accomplished 
> by augmenting such a parameter into the existing model, and save the 
> need to create YANG data instrumentation that is basically redundant, 
> hence reduce the complexity of implementations.
>
> --- Alex
>
> *From:* Tianran Zhou <[e-mail]>
> *Sent:* Thursday, November 17, 2022 10:49 PM
> *To:* Alexander Clemm <[e-mail]>; [e-mail]
> *Subject:* RE: Manifest need? Re: 
> draft-claise-opsawg-collected-data-manifest
>
> Hi Alex,
>
> I think this is a very good discussion. I raised this question before 
> in the mailing list.
>
> I think there may be some benefits:
>
> 1. The meta data can always go with the telemetry data, without 
> explicit subscription. This facilitates the close loop automation.
>
> 2. Another subscription to the subscription information may make the 
> management complex, since we put all the subscriptions in one list.
>
> Best,
>
> Tianran
>
> *发件人**:*OPSAWG [mailto:[e-mail] 
> <mailto:[e-mail]>] *代表 *Alexander Clemm
> *发送时间:* 2022年11月18日 7:40
> *收件人:* [e-mail]
> *主题:* [OPSAWG] Manifest need? Re: 
> draft-claise-opsawg-collected-data-manifest
>
> Hello draft-claise-opsawg-collected-data-manifest coauthors,
>
> I just wanted to follow up on my comment at the microphone in London 
> regarding draft-claise-opsawg-collected-data-manifest.
>
> Clearly, there is a need to preserve context to be able to correctly 
> interpret data after it has been collected.  That being said, as 
> currently stated, the draft appears to overspecify some of those 
> things, defining some YANG data that IMHO is not actually needed as 
> the same can already be accomplished.  This concerns the specifics 
> about the subscription itself.
>
> RFC 8641 includes a YANG model that reflects for each subscription how 
> it is configured (whether configured directly or established by RPC), 
> including the selection filter, the update trigger, the period and 
> anchor time (in case of periodic subscription), dampening periods and 
> excluded changes (in case of on-change subscription), etc. The 
> corresponding YANG data can be itself be subscribed to, or retrieved 
> on demand, just like any other kind of YANG data.
>
> I am therefore not quite sure what the proposed manifest would provide 
> that couldn't be accomplished already.  The suggestion to retain the 
> subscription data along with the subscribed data makes a lot of sense 
> but would appear to be a practice that will be up to the management 
> application to implement, with the mechanism already provided.  (This 
> could of course be included as a description of a recommended practice 
> in the draft.)  Or is there something else that is missing?
>
> If there is indeed a delta that cannot be otherwise accomplished, my 
> suggestion would be to add text to the draft that clearly describes 
> the possibility of subscribing to subscription configuration data, 
> then explaining the functional delta that your draft covers in 
> addition to that.
>
> Thanks
>
> --- Alex
>
>
> _______________________________________________
> OPSAWG mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/opsawg