Re: [netmod] Deviations and augmentations

Robert Wilton <rwilton@cisco.com> Wed, 14 November 2018 10:48 UTC

Return-Path: <rwilton@cisco.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ADE0112D4EB for <netmod@ietfa.amsl.com>; Wed, 14 Nov 2018 02:48:20 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.502
X-Spam-Level:
X-Spam-Status: No, score=-14.502 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rZshoI8ighnt for <netmod@ietfa.amsl.com>; Wed, 14 Nov 2018 02:48:18 -0800 (PST)
Received: from aer-iport-1.cisco.com (aer-iport-1.cisco.com [173.38.203.51]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 523C8124408 for <netmod@ietf.org>; Wed, 14 Nov 2018 02:48:18 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=3926; q=dns/txt; s=iport; t=1542192498; x=1543402098; h=subject:to:references:from:message-id:date:mime-version: in-reply-to:content-transfer-encoding; bh=wPMCC+jguMH2lRXej1H1fwnLEOA6Ic9iRpB6pvik+vo=; b=kcAmA2QM/wsOXjIGKXw57k003PnLAX2PAxxCAdtEQct/RyetI3BVmq6T CncyTpxLduTPga05h70xuHiYFRXJ7Qx4Xq+PbIZlBKiXFCf6gYbFnN7NB WQbi0di427AKgU7vylJit9XUOVwS2tvqMUp9SyFYA8WwTqk03ZZpx2T0K I=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A0AJAAAy/Otb/xbLJq1iGgEBAQEBAgEBAQEHAgEBAQGBUwMBAQEBCwGDa4QfiHeNBSWXNoF6DYRsAoNlNgsNAQMBAQIBAQJtKIU7AQUjDwEFPBULGAICJgICVwYBDAgBAReDBoICqBiBL4VAhG6BC4sRgUA/gTgMgl+EaIMaglcCiRMwlhsJkRwGGIlZhxyRKIZZgUwKJ4FVMxoIGxWDKJBZPwOOJQEB
X-IronPort-AV: E=Sophos;i="5.56,232,1539648000"; d="scan'208";a="8039940"
Received: from aer-iport-nat.cisco.com (HELO aer-core-1.cisco.com) ([173.38.203.22]) by aer-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Nov 2018 10:48:16 +0000
Received: from [10.63.23.62] (dhcp-ensft1-uk-vla370-10-63-23-62.cisco.com [10.63.23.62]) by aer-core-1.cisco.com (8.15.2/8.15.2) with ESMTP id wAEAmFdH026915; Wed, 14 Nov 2018 10:48:16 GMT
To: Andy Bierman <andy@yumaworks.com>, "netmod@ietf.org" <netmod@ietf.org>
References: <a8c912c8-a7a5-1852-d053-10f0f11076e8@cisco.com> <20181112.173351.1984161388756642220.mbj@tail-f.com> <cbe0103b-112e-4687-e119-0698ea6cb9f4@cisco.com> <77b69d64-2ce2-29d9-77a9-04a7159003a9@ericsson.com> <CABCOCHQmA1PaVTu7oLiECXLrCULqW1KJddDRvYaDmE4xWu5AmA@mail.gmail.com> <98d6293c-d762-4d21-a9e2-c9cb20f74135@cisco.com> <CABCOCHR-vygv+Fq8JWGMm59-V6CB4PkqfSA_5mR8xBUqwi6xDw@mail.gmail.com> <453368b2-aa52-f09a-ea0b-960255bce46b@cisco.com> <20181113170652.intfq37w6rxyw4rq@anna.jacobs.jacobs-university.de> <c6de1ed9-7dfb-fff1-0069-fa22dc2e3303@cisco.com> <20181113182621.vixdnijldgs7pmtj@anna.jacobs.jacobs-university.de>
From: Robert Wilton <rwilton@cisco.com>
Message-ID: <cf2b5769-5058-8728-f02d-8bae39ec715e@cisco.com>
Date: Wed, 14 Nov 2018 10:48:14 +0000
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.3.0
MIME-Version: 1.0
In-Reply-To: <20181113182621.vixdnijldgs7pmtj@anna.jacobs.jacobs-university.de>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Outbound-SMTP-Client: 10.63.23.62, dhcp-ensft1-uk-vla370-10-63-23-62.cisco.com
X-Outbound-Node: aer-core-1.cisco.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/TGmzDoqQlOCt80qina1hlebIIvE>
Subject: Re: [netmod] Deviations and augmentations
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 14 Nov 2018 10:48:21 -0000

On 13/11/2018 18:26, Juergen Schoenwaelder wrote:
> On Tue, Nov 13, 2018 at 05:54:27PM +0000, Robert Wilton wrote:
>>
>> I think that my main point regarding version numbers is:
>>
>>   - If we are going to use semver then we should define and follow the rules
>> strictly.  I.e. assuming that the semver has been populated correctly then
>> it should be possible to determine whether the change is a nbc, bc, or
>> editorial change.  I think that a solution where semver is done in a
>> half-hearted way is entirely useless (e.g. if clients cannot trust the
>> version number).  I think that there is a pretty clear requirement to
>> support NBC changes to older revisions (i.e. I really mean bug fixes but a
>> looser interpretation than your definition), and hence I don't think that
>> vanilla semver is sufficient.
> But even if you make the 'YANG world' use semvers correctly (not sure
> how you want to achieve that but lets pretend you manage), then the
> three digit number does not tell a client whether it will work or not.
> What should a client do if semver indicate nbc? What should it do if
> semver indicates bc? What should it do if semver indicates editorial
> change? Where is the "protocol specification" for this?

In theory, for editorial or bc changes an existing client, coded 
correctly SHOULD work unchanged (assuming that it doesn't barf on 
operational state that it doesn't understand or config that it isn't 
aware of)  I.e. I see that it is effectively making the same promise 
that 7950 module updates rules are trying to enforce today (putting 
status obsolete aside).

For an nbc change they would need to check, manually or using a schema 
compare tool, to see what has changed.

But I agree that a scheme level version number is really giving a 
suggestion rather than a robust promise that the client won't break.

Even the schema compare tool doesn't guarantee that a client won't 
break, since it does not guarantee that the server implementation 
doesn't have any bugs.

The next step is a solid set of tests that can be run against the 
updated schema and servers, and check that they behave as expected.  Rob 
Shakir indicated that OpenConfig is busy building such as test framework.

Then I think that the operator still rolls out the changes gradually, 
monitors for unexpected conditions, and is able to roll back if 
something goes wrong.

But none of these stages prove that the network will still be fine 
afterwards, they just help catch issues earlier, where they should be 
easier to handle, and give a bit more confidence that the previous step.


>   
>> - An alternative is to not use a semver version number at all, instead we
>> define a non semantic 3 digit version number that is something like:
>>
>>   major = significant changes to the module have been made
>>   minor = smaller changes/feature enhancements to the module have been made
>>   patch = a bug fix to the module has been made.
>>
>> Clients, module readers, etc, and then required to use a schema comparison
>> tool to determine what the actual changes between two revisions are and
>> whether or not they will be broken by those changes.
> The only robust solution is schema comparison and we may think about
> what kind of annotations make such schema comparisons easier or faster
> or how we can help clients to adapt (e.g., providing pointers where
> definitions have moved, providing machine readable information
> allowing tools to track which modules replace others etc.).

Yes, I think that all of this is a good idea.  But I still think that 
having a human readable version number per module is also helpful, even 
if it is only to help humans label the module revision in an easy way.  
E.g. I think that a version number is easier to parse/remember than a 
revision date.

Thanks,
Rob


>
> /js
>