Re: [yang-doctors] YANG dialects?

"Rob Wilton (rwilton)" <rwilton@cisco.com> Thu, 19 December 2019 17:18 UTC

Return-Path: <rwilton@cisco.com>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9D646120964 for <yang-doctors@ietfa.amsl.com>; Thu, 19 Dec 2019 09:18:50 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.5
X-Spam-Level:
X-Spam-Status: No, score=-14.5 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com header.b=W7GbZ8yy; dkim=pass (1024-bit key) header.d=cisco.onmicrosoft.com header.b=nMnx1oNL
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id J39JYI0JvJni for <yang-doctors@ietfa.amsl.com>; Thu, 19 Dec 2019 09:18:48 -0800 (PST)
Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AF68712092A for <yang-doctors@ietf.org>; Thu, 19 Dec 2019 09:18:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6044; q=dns/txt; s=iport; t=1576775928; x=1577985528; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=1TLJZJ9aXRpv5Ce6Ne4UDHvu9+pIOJAkui7dQRecp30=; b=W7GbZ8yynN7PiOtocmQiIEU3gXDGQ6BgNoiplC5NtnlSGXkXn9lwXWD9 +SUay5BL1ufzFEd4pWa1F7V6Lwd1fLMYncCZVY1NGnPDttX9aRPcuDqnC eEV/KLaI13CThxyFWQujhTaFGoz/fZf4dyCE2vqm0ZUs6GAPkpuP6+kNg s=;
IronPort-PHdr: 9a23:RhNWFxenYsvipsBzAi62Xpi1lGMj4e+mNxMJ6pchl7NFe7ii+JKnJkHE+PFxlwGRD57D5adCjOzb++D7VGoM7IzJkUhKcYcEFnpnwd4TgxRmBceEDUPhK/u/dTM7GNhFUndu/mqwNg5eH8OtL1A=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A0D1AQA+sPtd/4oNJK1cCRsBAQEBAQEBBQEBAREBAQMDAQEBgXyBTSknBWxYIAQLKgqHQgOKc4JfmAiCUgNUCQEBAQwBARgLCgIBAYRAAoIcJDgTAgMNAQEEAQEBAgEFBG2FNwyFXgEBAQECAQEBECgGAQEsCwEEBwQCAQgOAwQBAQEeECcLHQgCBAENBQgRAgeDAYJGAw4gAQIMoU4CgTiIYYIngn4BAQWFHxiCDAMGgTaMGRqBQT+BEUeCTD6CZAEBgTksg0CCCiKNQqFNCoI0ljOaUY5RmlECBAIEBQIOAQEFgWkigVhwFTuCbFAYDY0SgScBB4JEhRSFP3SBKI0vK4EEAYEPAQE
X-IronPort-AV: E=Sophos;i="5.69,332,1571702400"; d="scan'208";a="386198572"
Received: from alln-core-5.cisco.com ([173.36.13.138]) by alln-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 19 Dec 2019 17:18:46 +0000
Received: from XCH-RCD-010.cisco.com (xch-rcd-010.cisco.com [173.37.102.20]) by alln-core-5.cisco.com (8.15.2/8.15.2) with ESMTPS id xBJHIkuv006594 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 19 Dec 2019 17:18:46 GMT
Received: from xhs-rtp-002.cisco.com (64.101.210.229) by XCH-RCD-010.cisco.com (173.37.102.20) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 19 Dec 2019 11:18:45 -0600
Received: from xhs-rtp-001.cisco.com (64.101.210.228) by xhs-rtp-002.cisco.com (64.101.210.229) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 19 Dec 2019 12:18:44 -0500
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (64.101.32.56) by xhs-rtp-001.cisco.com (64.101.210.228) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Thu, 19 Dec 2019 12:18:44 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GWc2o4XBKGr/+LNfgDXMDjV7i5THGbwInviej0PZinQ9FfljQntQR4zMQTDgaQjUYWW4X86a6y081MYKRbE0WZ9TsS9xbFRWGmjG4glRsBbFkkUxGGedik2LbWQzXcigoL73qtg0+o2goGA2I5KMN+K1l6dZQwioAvSCprzrC37X9de9XPgwdd5FlCVZGTVU667RoSfR+nCaLNSRXqTEjk5uOASJqtm3vZSV2NjilxKkR1lohRvS0dgBDzeW01i5I5pefRnIEyFjxlFaUxisQpavoqoCiUABio3n3sIV4ShJBEakOusiMGcvOf114Q116FB8VsvuM4iN+2rMHb/OoQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OK5aAeF4n9vnVfowIpgpxJkh0E+ARykPa63f1JHTTKE=; b=ILF8y8Bq8bx33GmwikxVHOsOCsAsoQwHkH3xyWAHb6y9GExvSDGSPE7q1m0bCY6ZeQUCXVoq68UUdBbHhpp9JHjhzCqbDe3ktPukRHERyrQLKJM3dvYTKINrvzGiX5Kj+cgHzO6kpK9AtClBG4GcDX1PszXZs2TiM33LWW//S2GeJIWg6r1zeTLJWh9lqlcsR81FyHj0bcEgXDU67Plri4cZfoYCQRIX5STB1OGsdFYAv+pqQrp4TbMhfUlM63S/XUvaWK5k2Ao/P2KwFIfippHLaeSqebj9zrOFhTxrm5QARLunSneGGHwgqhovY0h/fJUvj+L460TmScCCtvE7Aw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=cisco.com; dmarc=pass action=none header.from=cisco.com; dkim=pass header.d=cisco.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OK5aAeF4n9vnVfowIpgpxJkh0E+ARykPa63f1JHTTKE=; b=nMnx1oNLVW3YHsh5owEaCYl5oyOjJP2d7dHSqcAbcFdDeyzxW29BLbo8zdrl1+NCTLFJwmWUaH3Y5CQO+EhmbBtWisjcEFYQ+IhMMLZC6+gYajE6j9UW9dghl8j6+FdbtcZK77oz30j890RGT1riPPGFo3AbOUZLEzU/3j3QHB4=
Received: from MN2PR11MB4366.namprd11.prod.outlook.com (52.135.38.209) by MN2PR11MB3663.namprd11.prod.outlook.com (20.178.253.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2538.14; Thu, 19 Dec 2019 17:18:43 +0000
Received: from MN2PR11MB4366.namprd11.prod.outlook.com ([fe80::8106:b538:2920:a44f]) by MN2PR11MB4366.namprd11.prod.outlook.com ([fe80::8106:b538:2920:a44f%5]) with mapi id 15.20.2559.012; Thu, 19 Dec 2019 17:18:43 +0000
From: "Rob Wilton (rwilton)" <rwilton@cisco.com>
To: Martin Bjorklund <mbj@tail-f.com>, "andy@yumaworks.com" <andy@yumaworks.com>
CC: "yang-doctors@ietf.org" <yang-doctors@ietf.org>
Thread-Topic: [yang-doctors] YANG dialects?
Thread-Index: AQHVtdlN9Q9/pQqZU06QplN4E6S++afAYPEAgAAgvYCAAJk9AIAAl5yA
Date: Thu, 19 Dec 2019 17:18:43 +0000
Message-ID: <MN2PR11MB4366F797234C963E23734BA1B5520@MN2PR11MB4366.namprd11.prod.outlook.com>
References: <CABCOCHTB+V6sV8hRcAC+OfseBBN=jUQnHQxzEVB_1drWhaEVXg@mail.gmail.com> <20191218.220130.545256750871048075.mbj@tail-f.com> <CABCOCHRfNrz+cNMA8ciRYGEM-7aR=S2NYJzU-t8JQ620Zs=hvQ@mail.gmail.com> <20191219.090708.1031388562992503497.mbj@tail-f.com>
In-Reply-To: <20191219.090708.1031388562992503497.mbj@tail-f.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=rwilton@cisco.com;
x-originating-ip: [173.38.220.52]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: a44195ea-c9c9-4f1f-24de-08d784a77fb7
x-ms-traffictypediagnostic: MN2PR11MB3663:
x-microsoft-antispam-prvs: <MN2PR11MB3663FAF4A7CE685C4433304AB5520@MN2PR11MB3663.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:7691;
x-forefront-prvs: 0256C18696
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(346002)(396003)(366004)(39860400002)(376002)(189003)(13464003)(199004)(51444003)(9686003)(66946007)(110136005)(53546011)(66476007)(66556008)(64756008)(66446008)(2906002)(4326008)(26005)(6506007)(71200400001)(7696005)(186003)(5660300002)(81156014)(86362001)(81166006)(478600001)(966005)(76116006)(33656002)(55016002)(52536014)(8676002)(8936002)(316002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR11MB3663; H:MN2PR11MB4366.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: cisco.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: 7FaaI59EQT41aUteuKqdrcyG5g6fodV+h1WdJGG8QF1KwJE4DQeFwDrCfI15iJX5/+3aSYbdl3XXLEak6KMK28szkN+GfrHb0MmP1DGTmC0WLPWnDPE9ge15CiAgGukFZ+alxVnx1pGp3wvspD3BHxRM5FkyLxQxGdXtbvLrxC2HtlOg/5btYm8vtrlWY4moiQWilOM6CQngo48WCGHNuWu7S8NdCgkfA2i/7Wjj5tPqEVeLY2+42H8d980oBr5KN3kSsn47hwMBYe6ysUnU8mqX9abbkRZ17kUwBOiwIACo2twWT2uv7C5oMRgNEw5JWVjEEuK5bVrwFKWD1iI12cra9b+mTkuHDady6JXxVg2A5u4DB8YqiBxfwHDGA+kd89ODfu9xie/q1YZ8mZL0ko9qWzDewAk28ndZmZCGsW7XVMxex+Jb18U4NhsLaKZyINjYGdO+fQBNcqpkoJUL6Y99fZyeYZXobApb/6vAZ+c87YTOzrwvzhcHJdsMKtqxmDiTOUyQ9BdUE6+3TGDOoh1MSXdaKNWs8yQTq5ilE/qhUveBXEOkZNbESWPmK61E
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: a44195ea-c9c9-4f1f-24de-08d784a77fb7
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Dec 2019 17:18:43.1549 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /VCyb5scR35UkbzKG0z57YFgiCE1Pb5I144vQ8U7rWdbClKIH/kMKkzDMvOgsJW/xdrVSZfzXFbM9Fe61iK5uw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3663
X-OriginatorOrg: cisco.com
X-Outbound-SMTP-Client: 173.37.102.20, xch-rcd-010.cisco.com
X-Outbound-Node: alln-core-5.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/5yubEN3GOk37npbyqrBky9YNklk>
Subject: Re: [yang-doctors] YANG dialects?
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 19 Dec 2019 17:18:53 -0000


> -----Original Message-----
> From: yang-doctors <yang-doctors-bounces@ietf.org> On Behalf Of Martin
> Bjorklund
> Sent: 19 December 2019 08:07
> To: andy@yumaworks.com
> Cc: yang-doctors@ietf.org
> Subject: Re: [yang-doctors] YANG dialects?
> 
> Andy Bierman <andy@yumaworks.com> wrote:
> > On Wed, Dec 18, 2019 at 1:01 PM Martin Bjorklund <mbj@tail-f.com> wrote:
> >
> > > Andy Bierman <andy@yumaworks.com> wrote:
> > > > Hi,
> > > >
> > > > I noticed in the help output for pyang 2.1:
> > > >
> > > >   --mef                 Validate the module(s) according to MEF
> rules.
> > > >   --ieee                Validate the module(s) according to IEEE
> rules.
> > > >   --ietf                Validate the module(s) according to IETF
> rules.
> > > >   --bbf                 Validate the module(s) according to BBF
> rules.
> > > >   --lint                Validate the module(s) according to RFC
> > > 8407rules.
> > > >
> > > >
> > > > I guess --openconfig is missing (since they have their own rules
> > > > for YANG as well)
> > > >
> > > > I hope these different rules are for style only, and not redefining
> YANG.
> > >
> > > Right, these are style rules.  Note the help text for "--lint" - it
> > > verifies the rules defined in RFC 8407.
> > >
> > > All the others invokes --lint, and in addtion validates module names
> > > and namespaces (e.g. --ietf checks that the module is called "ietf-*"
> > > ot "iana-*").
> > >
> > > > I suspect it is more than that.
> > >
> > > No, not for these validators.
> > >
> > >
> > OK
> >
> >
> >
> > > > I am aware of at least 3 openconfig issues that do not follow YANG
> rules:
> > > >    - repurposing pattern-stmt to use a different syntax
> > > >    - referencing config=false nodes from config=true context node
> > > > for when-stmts
> > > >    - use of identity strings without prefixes in XPath, where the
> > > > default module does
> > > >      not contain the identity (it is in an imported module
> > > > instead)
> > > >
> > > > Should we just continue hacking the tools in proprietary ways to
> > > > deal
> > > with
> > > > the inconsistencies? Are there any issues the IETF (or YANG
> > > > doctors) could or should address?
> > >
> > > These module are published and are being used.  Vendors that want to
> > > use them will have to work around the problems.  There's nothing we
> > > (the IETF) can do about them.  However, it is trivial to fix all
> > > these issues so that the modules are valid YANG, but it doesn't seem
> > > this is high on OpenConfig's priority list.
> > >
> > >
> > >
> >
> > Not so easy for all of them.
> > (1)
> > The pattern-stmt disaster cannot be addressed.
> 
> I think the solution is trivial, and it has been proposed to OpenConfig -
> introduce a new extension statement oc:posix-pattern.
> This will make the modules *legal*, and tools can use existing mechanisms
> to deal (or not) with this new statement.

But, as I understand it, the regex implementations are not POSIX either.  E.g. I think that they are using RE2 for Go, which is somewhat derived from PCRE which is derived from POSIX regex, but not exactly compatible either.

So, it would end up replacing one regex language that can't be 100% directly implemented with another regex language that can't be 100% directly implemented.  This seems to make the life more complex not less.

I still believe that the only solution is to define as subset of XML regex that can easily be translated into common regex engines.  Although, as Juergen has stated previously, this effectively defines a new regex language ...


> 
> > (2)
> > I see the config=false access issue in ietf-if-extensions@2019-11-04
> > and
> > ietf-if-l3-vlan@2019-11-04
> >
> >
> >       container dot1q-vlan {
> >         must
> >           'count(../../if-ext:forwarding-mode) = 0 or ' +
> >           'derived-from-or-self(../../if-ext:forwarding-mode,' +
> >                                 '"if-ext:layer-3-forwarding")' {
> >
> >
> > forwarding-mode is config=false
> >
> > I am trying to compile ietf-routing-policy and eventually ietf-bgp.
> > 3 different compilers are giving 3 different answers for
> > ietf-routing-policy, related to this dot1q-vlan container.
> 
> Hmm, I need to look at why neither pyang nor yanger detects this.  It
> should give a warning that the must expression always evaluates to true.

This is obviously just a bug that needs to be fixed!

Thanks,
Rob


> 
> > (3)
> > Users want to specify identities without the prefixes.
> > Should tools allow it or follow YANG rules?
> 
> Up to you.  But if the expression is on the form:
> 
>    "id-node = 'foo'"
> 
> and it should have been:
> 
>    "id-node = 'f:foo'"
> 
> 
> then it is probably not that easy to "just accept it", since the XPath
> evaluator will evaluate the LHS and get a string which is the string value
> of the "id-node" leaf.  Either this string value contains a prefix or it
> doesn't (7950 says it does).  Then it will evaluate the RHS and get the
> string 'foo', and then compare these two strings.
> 
> So if you change the string value of "id-node" to not contain the prefix,
> you may run into problems in other must expressions.  Or you do some other
> clever trick to handle this, but again, up to the tool imo.
> 
> 
> 
> /martin
> 
> 
> > From XPath POV it is just a string literal.
> > I know they are supposed to use derived-from (or-self) but they don't
> > follow that rule either.
> >
> >
> > /martin
> > >
> > >
> > Andy
> >
> >
> > >
> > >
> > > >
> > > > For starters, can somebody explain the different rules pyang is
> > > > checking here?
> > > >
> > > >
> > > > Andy
> > >
> 
> _______________________________________________
> yang-doctors mailing list
> yang-doctors@ietf.org
> https://www.ietf.org/mailman/listinfo/yang-doctors