Re: [netmod] IETF 108: Summary of insignificant whitespace changes and versioning

Ladislav Lhotka <ladislav.lhotka@nic.cz> Thu, 13 August 2020 09:37 UTC

Return-Path: <ladislav.lhotka@nic.cz>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 90C893A0A86 for <netmod@ietfa.amsl.com>; Thu, 13 Aug 2020 02:37:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.844
X-Spam-Level:
X-Spam-Status: No, score=-2.844 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.949, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IxlTOYHmpyZl for <netmod@ietfa.amsl.com>; Thu, 13 Aug 2020 02:37:23 -0700 (PDT)
Received: from mail.nic.cz (mail.nic.cz [217.31.204.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F68E3A0A84 for <netmod@ietf.org>; Thu, 13 Aug 2020 02:37:21 -0700 (PDT)
Received: from [IPv6:2001:718:1a02:1::380] (unknown [IPv6:2001:718:1a02:1::380]) by mail.nic.cz (Postfix) with ESMTPSA id C5A13140954; Thu, 13 Aug 2020 11:37:18 +0200 (CEST)
To: "Joe Clarke (jclarke)" <jclarke@cisco.com>
Cc: =?UTF-8?Q?Martin_Bj=c3=b6rklund?= <mbj+ietf@4668.se>, "netmod@ietf.org" <netmod@ietf.org>
References: <5CF24083-4126-4BE0-93F1-9A36F6DE9296@cisco.com> <20200811.164556.608015447238311339.id@4668.se> <A634B3C1-9F19-4A44-9479-56EC986DA1D8@cisco.com> <878sekb885.fsf@nic.cz> <11245BD3-6E79-4F02-9962-53BE87264460@cisco.com>
From: Ladislav Lhotka <ladislav.lhotka@nic.cz>
Autocrypt: addr=ladislav.lhotka@nic.cz; keydata= mQINBFhU5JIBEACmGIeLglGrwf6JBcNrcLoO19yAdKcH7V132STGHlhEAgmdzwclgVE7GN7y 5+1ySQ8jhDM80QSQ+XaXsSbHTQl23vcVvSfKNdme11kGEQ7kO/bTbUYxpRRa5dqDjLeEHhol WiNk6nUHqEAExbpEoITiO216Lwxh8gD2+39ZH/UVJgMHoZD1VmrxHSnp9b1SpmDWkbILRc93 u6lADieU67SG5vWqGaBgp8JQAVI3F+ZhcLXHQiQLPliePT3YKfvubWg9NIprts9fv2JAUywv NN4R5gg1oFegOXouzlBySiUNXndzJsj1JAcI4psGA9iGYBhgdILXg+2Kwkok9rrx1gWsqtmb lcFDiJRx52FUohHz9obZ9gkOd4NoV4jatjnu+9HKA8V2T5JUgTCgOWeI2yHoSNFJvKO0ygIg /5ccrB9iRdmJIiA2cAmu5MaHnKxAcQ7eaXqQN+JRcpFUH2ooFHKm0750U3aAZY3an5+a5Njh XddaDDX/7f0a68jaWuoKFn7Mqa2HhNnLJs2Nsq9quLdWEUh59wOgTrp2TZnkAFtG1MGa2dNz M3YX22+jwbnfv/U72fEu33juXfMLWfwzJEzQG47dnpmsArt8fM+atOvTSvkLQDNJkOSVZT8m NR5/OeIYfxhe3FUSrljRQBXN8ioVoiMtcCR8EKXCwCnja20vdQARAQABtChMYWRpc2xhdiBM aG90a2EgPGxhZGlzbGF2Lmxob3RrYUBuaWMuY3o+iQJUBBMBCgA+FiEEtr1TxpCtJF8Aid5X uPkrCKn3bGcFAl3k3q0CGwMFCQecs7wFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AACgkQuPkr CKn3bGeXZA/+IM3zgfkMwQ+rvYMEn1cHoYwA0j8iwGtQp6FQYO/Vj0d6Q07Q5/iUqM0UuniU HN6xC5APrPsxwyeSuJUW1zjGm6JD8U5WinbNJuBU/FMC7XyHgOwwXdlT1bLlhOYnvUrZx0bB QeMLnUinvTHLVFnpUCfFpyKRUKh3kkUJS5AXRz5ZgEqJUr79GSTg46dluXlz9mnPIkwluwYO ydlhtW/+s/xVuCIqRUdhhc30d8K5XB+smiYmHSzePjLGwsP1pHt1e8we57EFNvm7iiwXPCAI cy+PAtCxmq2zsIxsZxtZP4SWbRcj2Z16XTtT+hhNK6SWKhpDuacadpWChXpYc36xtuJ5Ll3C lZqa/6mDRcagJy016pQboW/4JIkFcFhu6ugNwvR1OBom04VPVBxqQRgazqpfkB0jSWWS4rwh l30kwDFW5eWcski3Ja5g2drj0YpUeEkMOQcToxLPXffI6dauKTMh7K6RF++LGCYxOukWt2r+ jk6/pB3rSVztAJDb4kc/EgE7yNwrVQOxkKHfk3rGCXkvijCgU3jS0k5g01/OwENxpEUk0Nln mlLJdb8Bju6Jp2y3ZX97+Szs2tH8M6Mpny213lFgc4EVCOFVtIPsyYfiR9On1IIdRqMx3mQC 5AV+3Wc0o3M7Ku0ZbuhkYLNIhlNrRjlxFRk12FpZwFwznY+5AQ0EWFTnYgEIAM6XJ37HGp7H nzztkWbCW+5zkLzff2LQh7yoH32CGJTK7vnQKOY+JKi16lR1qUDHOUtYfRj17jBNWaB0CY0k qkRWCDXG0pff8VT63Kg/DlS2LLbPQpO2NcjdMhPTTUadnKWeo7h6/mFF7VW4X2sm1NYeNWD5 VQxdPQUbnCJ4Pi23QZRJnBu2ycM9aXa7ERRvpN3i3mAc2UgkJWq4KGjI74cd7ytCsU0RN6RT S0sFE4zCB1r6nXqfHHiousATS6+3GnuIGTtaeR0nXHlIvl2AC91+hswsWaA7YB4dB2/GJ8PN L3d07gA6yR2DoZJZhAmtMUiUzap4jgf07UoTOY80MHkAEQEAAYkDcgQYAQoAJgIbAhYhBLa9 U8aQrSRfAIneV7j5Kwip92xnBQJcLzGrBQkHnLFJAUDAdCAEGQEKAB0WIQQ+THLKBBuTzVYa 3ozuYRDCyLOtngUCWFTnYgAKCRDuYRDCyLOtnhnlB/49PV5puyMGBJXY5XBdaPIKFfC5eBfy EOgMqDfJS9yhV7bhFRzzztX9Z7COod6mxmlogNaDFMk/yFH25DOe2tpjKs6sqa9bwyY3cOFw pp38VdeAq0FxF3qOhjKzF12RVQ4Ipu/ahzzQ6V5reia+onOghw+BrjOsPExJaDbiif5UqL/F yWvERlR46GGbFzK9gEvUo+52Wjx3t4hW2IMZKS58xvzkMkTQtxYHAM++jO7bH18wV603JJC3 mZxq9YKKFHVcxinp9l8yzVxfIjhiklednvb30lggY4VEG6X+BER/lcw1QgSDojjbHY72o+76 rBYZ+pSacY/idn5AkAEkQyxsCRC4+SsIqfdsZ1aAEACjuN1b73KNsdsNZuV/MSdmfC+LkoG2 +20kGWzHHiq1Owc4W3fDRohJThM5xyKt2RAcLZAqzfeef1AuoESTqqR6OheaF3u0HAa+U87E /dfjflBOsGZloNi/0/JcN2B8VJpXBwA55adA4YwNL/i8k/roIZ0Yh4wd8Bv8G4exrjvPZPCG klGNBvIJomaqgb9YDvdN22CW3f0zoCwdTX+XV7YL6IQsZroRViLWZv12UCKO1kVop8IMQolx XGbo4kVkqtVx0A4O+ZqCiG819QGZIAtaKSG6aK1xWxNg16ClbEoCEglPeLiOr1ttibO6emuK 4w+4vA5nu+q5niUZ611f/9OtnNO6w/EZ9jRjiqBbo7esNkzal4LjwZUa0B3poWRz+7B++CYS F0c/qdfqjYmiN4IEbXnC0EFhwOYejTayy/H/nA0yGjxF7xA2Krom8B3YTaOnaZ2lVx0gtQK6 Ih7SSc842yRyBnMoVko8mzcHb/0urD2uWkPI5VfXLF9OJhfe3J1xswtqfOxFtf1cZv6fnQID 4gfsvRP8s8Wow5m3gp1HLSlxeZcc57nK8/nKWaTssdOf6doSw8kLWC5Dcwx2Ld15SsjkHeCO MtOaIxq+Er6vnvcepj76pZZJznS/bMriYPyyw0FIYmAPcnm3v5W+HZ339ktp+Kk3fNl3qJo6 C+Sjv7kBDQRYVOgGAQgA0GW5sWxqhcD9hxi/LQZ/hzPBgrpTvyIgxHrJN1HZ2BzdpNU7UlnC DVDzvAV2/1kEIiCdTYnnly0+PCHhyEdP45M9//oH7KYd/F/JSQwI+56mGP2O0fBBu9WStHws RraGfzGGZDV1XutvIz/qzpkBGpXGzA73Bi2Xye+/9JtHXmjhoXnItQLO38Hqnb5kXk79cSC4 TStPAKdrB/CMolcArT8j9eijIUOPMkWCGCeICv/hECpA5CZma2tplU80cKiOIYknFG8E+5Pr FlW962P7njIaahWSIObM2/C48uu5KAwgyDj6DA14cwc/4wfRsXZFjUMNpI0xer1iLQV5y1vi 6QARAQABiQI8BBgBCgAmAhsMFiEEtr1TxpCtJF8Aid5XuPkrCKn3bGcFAlwvMc8FCQecsMkA CgkQuPkrCKn3bGfkEg//alC3LNxnyH3tNy/OKQqOFJLUCidIm77DT/ZQfjLMAZezzCAq1Adf hVg7uV5a/dFdQdVDnjEz+SQ6/EzyqGashdAMnlBDwiqPe3MhB0L97VQwjNQa0wf1AsHdQCv8 i3e7U6RjlEmSC42dA+RjhrFOCZPtUTlCVDoSnY1S0kZOvEjhHvw901MPEINikZY4TBc18Xnk fbGQhglvI9NqBawlBKHyDGTAVJL7ld60iHsWt3prq+h48iFjc7x2JT39oBxT5Jhl3pg01QSC vu2Fg99vUuRr3u8tavnMKX4Vk3bujpHhuyOTSm0X37nwwu9DpMqkeUNvYR2RM+EuGGltonrN yrb5S21omN7fLDvRtdUBUOCFJLxZunTDLlkig/QuT+QlW7CG59Ussuu37HxgpiICCeeBDOaF bkZku43Qwt5cTtdJ8BkzYgvvoC7NegBn9N+U+Etrrvs09Slo11wlQx1F3sk9T9rjz1QrPM8j blxEgepm+W75VJ22z1qY/JnNElygR7KeTaeOY+iCn4wgDeC964AHMuGzG7aUg8P3zjgClL1T pFJwQUml880L6C+VuWPxuFIS18BPCYtLGlfQc5jp4dGN87ftenT13W70XiGXdrmIAFBF6Jjk 3huuDoRvdbPavAGdxu5QGdHeH0/GjfTxq7lRwgamoY0lw1LIB6LOzXo=
Organization: CZ.NIC
Message-ID: <acfe1b95-e0f3-0b7e-2635-9582eb11b4e6@nic.cz>
Date: Thu, 13 Aug 2020 11:37:18 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0
MIME-Version: 1.0
In-Reply-To: <11245BD3-6E79-4F02-9962-53BE87264460@cisco.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-Virus-Scanned: clamav-milter 0.102.2 at mail
X-Virus-Status: Clean
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/9o4BuG8Kun8cq1yc3hOOvdU3y-o>
Subject: Re: [netmod] IETF 108: Summary of insignificant whitespace changes and versioning
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 13 Aug 2020 09:37:27 -0000


On 13. 08. 20 10:52, Joe Clarke (jclarke) wrote:
> 
> 
>> On Aug 12, 2020, at 04:04, Ladislav Lhotka <ladislav.lhotka@nic.cz> wrote:
>>
>> "Joe Clarke \(jclarke\)" <jclarke=40cisco.com@dmarc.ietf.org> writes:
>>
>>>> On Aug 11, 2020, at 10:45, Martin Björklund <mbj+ietf@4668.se> wrote:
>>>>
>>>> Hi,
>>>>
>>>> "Joe Clarke \(jclarke\)" <jclarke=40cisco.com@dmarc.ietf.org> wrote:
>>>>> At the IETF 108 virtual meeting, Lada asked about what would happen if
>>>>> he converted a YANG module to YIN syntax (or vice versa, or to some
>>>>> other format).  This was during the discussion of the issue of what
>>>>> should happen if a module changes and the only changes are
>>>>> insignificant whitespaces (e.g., strip trailing spaces, change line
>>>>> length of descriptions, etc.).
>>>>>
>>>>> The authors/contributors discussed on this on our weekly calls, and we
>>>>> propose:
>>>>>
>>>>> If a module changes and those changes are only insignificant
>>>>> whitespace changes and the syntax of the module remains the same
>>>>> (i.e., YANG to YANG, YIN, YIN, etc.), a new revision of the module
>>>>> MUST be created.  If you are using YANG semver as your revision
>>>>> scheme, you MUST apply a PATCH version bump to that new module
>>>>> revision to indicate an editorial change.
>>>>>
>>>>> The reasoning behind this decision is that it makes it very clear and
>>>>> unambiguous to consumers that this module has been consciously
>>>>> changed, and those changes are only editorial.  This way one won’t be
>>>>> concerned if they note that a module of a given syntax with the same
>>>>> version but different checksums and diffs wasn’t otherwise
>>>>> manipulated.
>>>>
>>>> I think this is the wrong way to go.  I clean up formatting issues all
>>>> the time, including IETF modules.  I am pretty sure that if you
>>>> retrieve modules like "ietf-interfaces" or "ietf-yang-types" from
>>>> different vendors' products, you will get modules with differences in
>>>> whitespace - and this is not a problem AFAIK.
>>>>
>>>> I think it is ok that a simple "diff" show whitespace changes in this
>>>> case.  I don't think it leads to any real problems.
>>>
>>> We discussed this on the call.  The thinking was that a long diff output would essentially be unwieldy for some modules and important changes might be lost.  If the versions were the same, it would be ambiguous to the consume as to whether or not the module was only changed in trivial (i.e., less-than-editorial) or if more substantive changes happened.  If you trust the producer, maybe you assume they regenerated the module without trailing whitespace (or the like).  We felt there should be a more explicit signal.
>>>
>>>>
>>>>> That said, if a module changes format from one syntax to another but
>>>>> maintains semantic equivalency, then the revision and YANG semver MUST
>>>>> be the same.  In that case, one will use the extension to realize that
>>>>> this module file cannot be directly compared to one of another syntax
>>>>> without looking at compiled or semantic representation.
>>>>
>>>> This seems a bit inconsistent.  Suppose I round-trip from YANG to YIN
>>>> to YANG, and the result is different whitespace in the two YANG
>>>> modules.  The revision is the same, as explained above.  How is this
>>>> different from changing the whitespace in YANG directly?
>>>
>>> We didn’t discuss this directly, but we did discuss auto-generators that could do this type of round-tripping.  The general consensus was that you would use the same post-processing tool (e.g., pyang -f yang) on the result to ensure consistency.  And a consumer would look to a canonical source (like IANA, the IETF document, or the vendor) to ensure a consistent module.
>>>
>>> In terms of alternate sources, I would think that if one wanted to trust an IETF module downloaded from some other site, they could.  If that site did some additional formatting, that would be up to the consumer to resolve compared to what might be required by a package.  But if the publisher (IETF in this case) were to republish a module with these stripped whitespace line endings, then that would be a different revision.
>>
>> I think it would be better to define "canonical YANG". One relatively straightforward way might be to convert to YIN first and then apply XML canonicalization:
>>
>> https://www.w3.org/TR/2001/REC-xml-c14n-20010315
>>
>> As an additional benefit, this would also enable digital signatures of YANG modules.
> 
> This came up on our last call as well, but the consensus there was that defining canonical YANG would be a large undertaking and out of scope for this work.  That said, I think codifying such things would be useful.

I did a few quick tests, and it seems that a pipeline like

$ pyang -f yin ietf-inet-types.yang | xmllint --c14n - | sha256sum
8d1ca8f30566ce8cbeffa095e20642f8f6e9f3a724286be4ead863b4467dc40b  -

might be a very good start. It is certainly much more robust than
relying on a simple checksum of the YANG module text.

Lada

> 
> Joe
> 

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67