Re: [netmod] yang-data-ext issues

Kent Watsen <kwatsen@juniper.net> Tue, 29 May 2018 21:20 UTC

Return-Path: <kwatsen@juniper.net>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EE94212EB92 for <netmod@ietfa.amsl.com>; Tue, 29 May 2018 14:20:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Level:
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=juniper.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SN2SNdK-zCyr for <netmod@ietfa.amsl.com>; Tue, 29 May 2018 14:20:39 -0700 (PDT)
Received: from mx0b-00273201.pphosted.com (mx0a-00273201.pphosted.com [208.84.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D5F1312EB19 for <netmod@ietf.org>; Tue, 29 May 2018 14:20:39 -0700 (PDT)
Received: from pps.filterd (m0108157.ppops.net [127.0.0.1]) by mx0a-00273201.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4TKxbZ7028225; Tue, 29 May 2018 14:02:23 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=juniper.net; h=from : to : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=PPS1017; bh=JUPJUobgXW51z8j3qHsBLtdLC1gnGM1wxv+qKn1QpMY=; b=GKAWNRhAiMLPU0ZnJpEj8cDzxP1rScRi8lEFlC0adLW9wqng4lZa+EFkUNK+6zRh/h/M BAdrbOoEEOAi24iDvKyB1HWQe4qIrA6sAry/Mohn0hw6U43W3Th6AZYe12o/+UORArgO opfA9ovA94CLw/0d6QhrP4LakO8+M4OjBb5o7O8CV2kLLdwACLSXSMNHRYJeEi57ajEr Y/k5GaOfJxQ4XW637nmd7cu5WhL6AzbVKlcVdJcNPJ0mvm55hsGD7d+h/Ltq2dWCwyK7 loKP4b3FvRbpLYjNMRmi5OQU5GNDDLPKOstB1NxcISaBQs5m8qg2SpAZM72CfDoqSbgW /g==
Received: from nam01-sn1-obe.outbound.protection.outlook.com (mail-sn1nam01lp0113.outbound.protection.outlook.com [207.46.163.113]) by mx0a-00273201.pphosted.com with ESMTP id 2j9cx38dk9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 29 May 2018 14:02:23 -0700
Received: from BYAPR05MB4230.namprd05.prod.outlook.com (52.135.200.153) by BYAPR05MB3959.namprd05.prod.outlook.com (52.135.196.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.820.5; Tue, 29 May 2018 21:02:21 +0000
Received: from BYAPR05MB4230.namprd05.prod.outlook.com ([fe80::95f0:e564:96c8:7f1c]) by BYAPR05MB4230.namprd05.prod.outlook.com ([fe80::95f0:e564:96c8:7f1c%2]) with mapi id 15.20.0820.010; Tue, 29 May 2018 21:02:21 +0000
From: Kent Watsen <kwatsen@juniper.net>
To: Andy Bierman <andy@yumaworks.com>, Juergen Schoenwaelder <j.schoenwaelder@jacobs-university.de>, Martin Bjorklund <mbj@tail-f.com>, "netmod@ietf.org" <netmod@ietf.org>
Thread-Topic: [netmod] yang-data-ext issues
Thread-Index: AQHT1YJYRSnZmZlnhU6JIK4LoPfNTqQEcM6AgAYAzwCAAlYhgIAB08UAgAAyUgCAATikgIAAAaSAgAGHGgCAAAK9AIAAEH6AgAACpYCAAPWtAIABPh0AgACT0YCAAAY/gIAABasAgAALQACAAARkgIAANngAgAADfgCAAAdSAIAADKEAgASYmICAAD0hgIACdpWAgAAbLACAAAZCAIAAAyoAgCqWpwCAAFa+AIAAJP2A///ZJgA=
Date: Tue, 29 May 2018 21:02:21 +0000
Message-ID: <21C39230-92EC-4BB5-82D0-2598300B362C@juniper.net>
References: <CABCOCHSupojOLssLebB-mR_PybRLA_4bcbaNF6-8ZUrx1Pu53w@mail.gmail.com> <20180502.092527.2305319833268262996.mbj@tail-f.com> <9fca04b0-fb29-36b3-67aa-2f2c4fb98748@cisco.com> <20180502.112506.845305331945500257.mbj@tail-f.com> <20180502093626.ugsg6nq24a6vjtdn@elstar.local> <64990DFB-CF50-401A-A4EF-B6161C8D227B@juniper.net> <20180529170900.qboedr2rmefsmblc@anna.jacobs.jacobs-university.de> <CABCOCHQ4DJgGg4WQJRhqKAAjoX=DDPHNavTB5WQVVdzLmXJZRw@mail.gmail.com>
In-Reply-To: <CABCOCHQ4DJgGg4WQJRhqKAAjoX=DDPHNavTB5WQVVdzLmXJZRw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/f.20.0.170309
x-originating-ip: [66.129.241.12]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BYAPR05MB3959; 7:eH9Y1vzubY6oMF3waKU8afMq0UtuPPYs8bXOY+xhtXYeSGKjtbq3R/bjmY74mT3dcQIfMMLp+J+tmv6oUKfEmx0/HEzJ6CkCMX4Oo3M1f4B560hZn7TDtElo16mtYqSgJXenxgdb0afEE1VXFAeI8HRUIS5P64ko8PmxB48jzep0OITmxaNkmiJm+BPNUwWdm/OVCVzqYwYq/evWVzI7r2iOH10wzQbQK90km5p05LsztxxozWWC7sdt7G+WkiiH
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(48565401081)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:BYAPR05MB3959;
x-ms-traffictypediagnostic: BYAPR05MB3959:
x-microsoft-antispam-prvs: <BYAPR05MB3959A0FEEDA8B1C6127314DBA56D0@BYAPR05MB3959.namprd05.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:;
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3231254)(944501410)(52105095)(3002001)(93006095)(93001095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123564045)(20161123560045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:BYAPR05MB3959; BCL:0; PCL:0; RULEID:; SRVR:BYAPR05MB3959;
x-forefront-prvs: 0687389FB0
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(376002)(39860400002)(366004)(39380400002)(396003)(346002)(199004)(189003)(99286004)(102836004)(6506007)(59450400001)(83716003)(478600001)(82746002)(8936002)(68736007)(966005)(36756003)(316002)(26005)(93886005)(186003)(446003)(476003)(86362001)(6486002)(76176011)(2616005)(6436002)(11346002)(110136005)(58126008)(486006)(229853002)(97736004)(5660300001)(5250100002)(2501003)(66066001)(33656002)(6306002)(2900100001)(3280700002)(25786009)(2906002)(5890100001)(6246003)(14454004)(53936002)(3660700001)(105586002)(106356001)(3846002)(305945005)(6116002)(6512007)(7736002)(81166006)(8676002)(81156014); DIR:OUT; SFP:1102; SCL:1; SRVR:BYAPR05MB3959; H:BYAPR05MB4230.namprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: juniper.net does not designate permitted sender hosts)
x-microsoft-antispam-message-info: xIZS2U4dzJEFg9rPkDQkh/l8/0w3JFll5hpKsDNqwayUcTHgIxfteGkJeSADmtK0B4XyosXIp6VHOSO/D8r+rMqNcH1Y3NgUmmeh+yWJS8w8fOOSp0XsQGYHk7hj/VMBShEcnYjkH2/9uRiCaXmapK+N5Y3dNMTEUkpJ6X3Foz0flwwJ5CkmkuagjPrDahtQ
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-ID: <D8A721A6EDD74948B7F3E32F44FB5637@namprd05.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: fe8b8bf6-aa10-4617-dbfa-08d5c5a7785f
X-OriginatorOrg: juniper.net
X-MS-Exchange-CrossTenant-Network-Message-Id: fe8b8bf6-aa10-4617-dbfa-08d5c5a7785f
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2018 21:02:21.1460 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR05MB3959
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-29_09:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1805220000 definitions=main-1805290222
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/z1UseEX6QurA-cJYvpzvNSYMI3Y>
Subject: Re: [netmod] yang-data-ext issues
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 29 May 2018 21:20:43 -0000

>>> >> The primary use case is not "generic RPC messages", but standalone
>>> >> instance documents, error-info structures, etc.
>>> >
>>> > The proper solution for rpcs and actions is to define error
>>> > information as part of the rpc/action. YANG 1.1 does not support
>>> > this but this is where it should be fixed.
>>> 
>>> Agreed, but note that the subscribed-notifications draft (both the
>>> published -12 and unpublished -13) are relying on being able to do
>>> just this, and YANG-next is years away...
>>
>>There is a description statement.
>>
>
> Automation tools generally use machine-readable statements, and avoid
> using description-stmts.
>
> The yang-data extensions do not solve this "rpc/action to error-info"
> mapping problem, so this problem is not a real factor here.

If subscribed-notifications doesn't use "yang-data", then it would need
to use another extension statement, lest the 3 "error-info" statements
it defines are misconstrued as configuration or operational state.


>>> > Standalone instance documents (not tied to datastores) may have their
>>> > use cases as well but it feels odd to create support for standalone
>>> > instance documents as extensions and then to create even more
>>> > extensions to support augmentation of these instance documents and
>>> > whoever knows what comes next.
>>> 
>>> What feels "odd" about this?  Is it not using the extension statement
>>> as it was intended?
>>
>> For me, extensions that define new data definition statements are
>> borderline. RFC 7950 has this nice statement:
>>
>>   o  extension: An extension attaches non-YANG semantics to statements.
>>      The "extension" statement defines new statements to express these
>>      semantics.
>>
>> This does not help since we lack a definition for 'non-YANG semantics'
>> and yes I know that yang-data is today defined as an extension. But
>> for me, this is a hack and instead of creating a slightly more
>> generalized version of this hack, I prefer to stick to yang-data in
>> favor of a proper solution as part of YANG.
>
> The yang-data and augment-yang-data extensions do not re-define YANG 
> data nodes. They define a new context for data nodes and prevent plain
> YANG tools from confusing this new context with data for configuration or 
> operational datastores.

Yes.

 
>>> > For short-term needs, there is yang-data defined in RFC 8040.
>>> 
>>> To be clear, the "short-term needs" are:
>>> 
>>>   a) zerotouch: to define a standalone instance document
>>>   b) notification-messages: to define a new notification message
>>>   c) subscribed-notifications: to define error-info structures
>>> 
>>> As I recall, this draft (not RFC 8040) is needed:
>>> 
>>>   - for (a), because rc:yang-data doesn't support a top-level
>>>     "choice" statement spanning "container" statements.
>>
>> So create a container.
>
> I tend to agree that a top-level choice could be avoided.
> This is not a really compelling use-case.

From https://mailarchive.ietf.org/arch/msg/netconf/T6OgCvbF30EOoU27b3T_-dOdj2E:

  The zerotouch draft has way too much text invested in the "zerotouch-
  information" artifact being polymorphic in nature to seriously consider
  having two unrelated artifacts, when the easier answer is to agree 
  that, in this case, the yang-data does in fact contain "data definition 
  statements that result in exactly one container data node definition".
  We don't even need to change any text, in this document [zerotouch] or
  in RFC 8040, in order to have this agreement.  We would only need 
  `pyang` to stop flagging it as an error.

So, yes, yang-data-ext isn't needed for zerotouch, there are options
on the table.  But, so long as this draft is a WIP, then it seems that 
zerotouch should use it.


>>>   - for (b), in order to augment a base yang-data "message" 
>>>     structure with additional nodes.
>>
>> So you are creating another augmentation mechanism. I am concerned
>> about ending up with a zoo of different mechanisms if we go down this
>> path, we may end up with every project or vendor creating their own
>> variants.
>>
>> With NMDA in place, YANG 1.1 is decribing schemas for datastores plus
>> operations and notifications. It is not a protocol message description
>> language or a standalone file format description language. If this is
>> needed, I prefer to create YANG X.Y - and if we manage the complexity
>> we have something that is ideally integrated and consistent.
>
> Yes it is a protocol message description language, but only
> in an incomplete, hacky way.  Specific subtrees within protocol messages
> are defined as special YANG data-defined portions and the rest is somehow
> out of scope for YANG. The way nested roots (e.g., <config> or <data>)
> are handled is the worst hack of all.  But IMO a new version of YANG is
> needed to really fix these problems.

Yes, we agree that a new version of YANG is ideal, but such is years away
and the NETCONF WG is not going to block these drafts waiting for it.  Do
we rather the notification-messages draft define a private version of the
"augment-yang-data" extension statement?


>>>   - AFAIAA, RFC 8040 is sufficient for (c)
>>> 
>>> Has anything changed?   I don't think that we can un-adopt this
>>> draft with said dependencies, right?
>
> I am just voicing my opinion. It may very well be that the WG prefers
> to go the route of not touching YANG 1.1 and instead patching around
> its limitations with extensions.
>
> My concern is simply driven that some want to patch in via extensions
> support for describing protocol messages and standalone documents,
> others want to patch via extensions and updates a different versioning
> system, and who knows what comes next. In the long run, I am afraid
> this will become a mess. And yes, it is always difficult to predict
> the future - we need crystal balls. Perhaps as an extension. ;-)
>
> This is a separate (but important) topic -- YANG 1.1 + a vendor-selected
> set of optional extensions is not the same as a new version of YANG 
> (with mandatory-to-implement statements).
>
> YANG designers cannot really rely on optional extensions across tool-chains.
 
True, but it seems that this one might hit critical mass. Even if it doesn't,
scripts can auto-convert yang-data definitions into their descendent node 
definitions (container, or choice in zerotouch case) so existing tooling can
grok them.  I have scripts doing this today.


Kent