Re: [netmod] Questions on coordination of models between standards orgs

[Jan Lindblad <[e-mail]>]

Rodney,

Excellent questions; I find these sort of questions come up frequently when discussing with SDOs taking up YANG. I'm not aware of much authoritative material from the WG on the topic, but here is my opinion, which I have shared with a couple of other SDOs in similar situations (with varying degrees of uptake). [JRL] inline below.

> Hello YANG doctors/experts,
>  
> I'm hoping to get some opinion on the best ways to coordinate development of YANG models among multiple standards organizations (e.g., IETF, IEEE, ITU-T, SMPTE <https://www.smpte.org/>).
>  
> As a reference for my questions, let's say that we have two standards organizations, one that specifies a "base" standard, and another that specifies a "profile" of that base standard. The base standard has a core set of required specifications, and a large set of optional specifications. The profile standard uses the base standard as its foundation, and it a) selects a subset of optional specifications from the base that are required for the profile, and b) adds its own unique specifications (some optional, some required). The engineers in each standards organization work on their documents independently. The profile standard uses the latest published revision of the base standard, but otherwise there isn't much explicit coordination.
>  
> In the context of YANG, the base standard organization will have its own timeline for YANG module revisions, and the profile standard organization will have an independent timeline for YANG module revisions. Each YANG module revision will align with a revision of the organization's published specification document (e.g., IETF RFC, IEEE standard in PDF).
>  
> The preceding description can apply to almost anything, but a specific example might help to understand. An example of a base standard is IEEE 1588 (Precision Time Protocol), which has a YANG model for its 2008 revision (RFC 8575). There are a large number of standards organizations that specify profiles of IEEE 1588, including ITU-T, IEEE (e.g. IEEE 802.1AS, IEEE C37.238), and SMPTE. Many of these profile organizations are currently planning YANG development for their 1588 profiles.
>  
> Question 1: What are the best mechanisms in the YANG data modeling language to use for a profile standard?

[JRL]
Pointing out modules and features by specification:
+ Profile SDO specifies set of Base SDO modules as required, and possibly additional sets that are conditionally required if certain functionality that is optional in the Profile is implemented
+ If Base SDO modules contain if-feature statements, Profile SDO may declare some of them required

Importing modules (or not):
+ Profile SDO may point out required modules without ever importing them -- they just need to be implemented (but this is uncommon)
+ Profile SDO modules may import Base SDO modules to access types or groupings (import only). If so, profile SDO should be clear about this intent.
+ Profile SDO may import Base SDO modules (implement).

Extending Base SDO module:
+ Profile SDO may author their own module tree (which typically references Base SDO tree, or types or groupings)
+ Profile SDO may use augment statements to inject additional model content in Base SDO tree
+ Profile SDO may use YANG mount points for Base SDO tree(s)
+ Profile SDO may want to use the YANG extension keyword to declare additional properties on the content

Equally important is what *not* to do. Here are some fairly common ideas that I believe are bad:
+ Profile SDO declares something that is required to implement in Base SDO tree as not required to implement in Profile
+ Profile SDO declares something that is mandatory to configure in Base SDO tree as not mandatory to configure in Profile
+ Profile SDO declares something they augmented as mandatory to configure
+ Profile SDO uses deviations in any way

All of these ideas violate the original work in the Base SDO. If the intent was to make things fit better together, not build silos and reuse existing code out there, that backfires badly when breaking the Base SDO modules like this. As Profile SDO, rather than setting or dropping requirements, state what happens if the operator does not create the optional augment, e.g. "if augmented leaf X is not set, feature Y will be disabled."

Deviations, when used by SDOs, are very destructive. First, interoperability suffers badly. What if an implementor wants to conform with both SDOs, where one SDO says it's got to be like this and the other says, no deviate like that? Secondly, it wrings a tool out of the hands of the implementor. There is no way in YANG to deviate a deviation. So if the Profile SDO uses deviations, implementors have no way of declaring the deviation they may have made from the Profile SDO.
 
> My assumption is that "import" and "augment" provide many benefits. The YANG modules for the profile can "import" the YANG of the base standard. This provides the core specifications from the base, as well as the base features that the profile is using (item a) above). The import also includes some nodes that the profile is not using, but since those nodes are optional in the core standard, they will not be marked as "mandatory", so the profile implementers can ignore them as desired. As for the new specifications in the profile (item b) above), "augment" can be used in the profile's YANG to add new nodes to the base tree.

[JRL] Yes, import and augment are prime vehicles for this.

Note that something that the Base SDO models as an optional leaf (a leaf without a mandatory statement) is not optional to implement. The absence of the mandatory keyword means it's optional for a client to configure, not that it's optional to implement. An element is optional to implement if there is an if-feature statement on it. The other way to legally not implement a leaf is to not claim to implement the entire module it is found in.

> As an alternative, the profile YANG could copy the contents of the base YANG into its own module, and edit from there (without import). The disadvantage of that technique is that the base nodes in the profile's YANG are more likely to diverge from the base standard's YANG over time. In addition to coordination problems between standards organizations, that can create challenges for implementers that are trying to support both YANG models.

[JRL] I have often encouraged people to "steal" from other organizations, but I have stopped to express myself like this, because some people may have interpreted that as encouragement to copy-paste parts from other organizations. I'm strongly against that. Apart from being a copyright infringement, it's also a bad habit for the reasons you state above. It gets messy and bloated quickly.

My real intent when saying "steal" was rather to encourage learning good YANG design by studying prior art.
 
> Maybe there are other alternatives? Maybe there are other aspects of the language that would help?
>  
> Question 2: Assuming the import/augment technique is used, how do standards best handle a node that migrates from profile YANG to base YANG?
>  
> As an example, assume the base standard has YANG for 2021, and the profile standard has YANG for 2022 (which imports/augments base 2021). The 2022 profile YANG has an augment that adds a "performance-monitoring" container to the base tree. Later in 2023 the standards organization for the base decides that "performance-monitoring" is a great feature, so the base incorporates "performance-monitoring" into the base standard. The base YANG for 2023 is published with "performance-monitoring" exactly as it is in the profile's 2022 YANG (same location, exact same container and leaves, etc). In 2024 the profile's standards organization is working on the profile YANG, so the profile's revised YANG imports the base YANG from 2023.
>  
> What should the 2024 profile YANG do with the profile's "performance-monitoring"?
>  
> Is it sufficient to mark "performance-monitoring" with obsolete status in the 2024 profile YANG, so that "performance-monitoring" in the base 2023 YANG replaces it?

[JRL] If the Profile SDO uses import by revision, which they should, I don't think this will be a problem. The base.yang module has rev 2021. The profile.yang has revision 2022, and imports base.yang with revision 2023. Next year, base.yang comes out in revision 2023. Nothing happens because profile.yang still imports base.yang revision 2021. At some later point profile.yang might be updated to import base.yang revision 2023, at which point the now superfluous definitions have been removed.

The current YANG versioning rules (RFC 6020 sec 10 and RFC 7950 sec 11) do not allow removing any content from a module, so the change above would clearly be a backwards incompatible change. This is true even if the base.yang copy-pasted the profile.yang text, since all the content switches from one namespace to another by this maneuver. Highly incompatible.
 
> As an alternative, in 2023 should the the base standard rename the container to avoid all collisions/confusion, such as "performance-monitoring-v2"?

[JRL] I think this use case is rather uncommon (corner case), but I would vote against using a -v2 suffix, unless the changes are a kind of 2.0 conceptual retake on this whole feature. As mentioned above, the namespace would shift anyway, which would be a rather dramatic change in all programmatic APIs, I think.

Best Regards,
/Jan Lindblad, YANG Doctor