[DNSOP] raising the bar: requiring implementations
Job Snijders <job@ntt.net> Sat, 24 March 2018 11:08 UTC
Return-Path: <job@instituut.net>
X-Original-To: dnsop@ietfa.amsl.com
Delivered-To: dnsop@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B68C1241F8 for <dnsop@ietfa.amsl.com>; Sat, 24 Mar 2018 04:08:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.67
X-Spam-Level:
X-Spam-Status: No, score=-1.67 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, UNPARSEABLE_RELAY=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UBk8xBMCNjNx for <dnsop@ietfa.amsl.com>; Sat, 24 Mar 2018 04:08:10 -0700 (PDT)
Received: from mail-wm0-f51.google.com (mail-wm0-f51.google.com [74.125.82.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 29E601201FA for <dnsop@ietf.org>; Sat, 24 Mar 2018 04:08:10 -0700 (PDT)
Received: by mail-wm0-f51.google.com with SMTP id a20so10187011wmd.1 for <dnsop@ietf.org>; Sat, 24 Mar 2018 04:08:09 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=oFzz/d2lGnJTkUIepQo5v/s93n+8gYQI5LOfwGkO1wg=; b=f16hOu05dJGtgIMsWzvh/RJ1oaGpk+qLK1iC9TDL4ARoOr9KASKUqIJWMzqxDPU1/I HRbXzNYSteZA9Io2rimTJjQQ9YIt2n+IOrILlxNr1PJRztc6sP8GoChl3IwFT+v63RgT omAZA+WsZsb53pOQVU8h+rAHegWLBfIP2wnR+dHO1x3+ufOkxRY4lYpuZwmdU5Jit5ZA Vc8fkwrH8EmGQwbMtlasvqX8qR2HqfzKVJD9jq/1pz0cn+E6djoU275qSusFHJxCj/B6 ZzknYZm4mGPAXVAynIFSzv3wHr0whAScDXIQBs6UUdSlPAO5YvMh4chkTjRqki+6mJzN owig==
X-Gm-Message-State: AElRT7GItmNCnMYlSHp57rKqmHRK3pBlG1F2Psr0qUeu3XYOzHu54Bbs acMZ48/RFK24+kvAmzvHHEQJpEYKF/8=
X-Google-Smtp-Source: AG47ELufm0wFTPqkuSava5hb1MKmUNs7jLem+x57RRLy6u9JEwM5zwE4qYBd3AgqLa7Fj1kBg4SwOQ==
X-Received: by 10.80.224.205 with SMTP id j13mr33668048edl.304.1521889687890; Sat, 24 Mar 2018 04:08:07 -0700 (PDT)
Received: from vurt.meerval.net (vurt.meerval.net. [192.147.168.22]) by smtp.gmail.com with ESMTPSA id o60sm6477075eda.17.2018.03.24.04.08.03 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 24 Mar 2018 04:08:07 -0700 (PDT)
Received: from localhost (vurt.meerval.net [local]) by vurt.meerval.net (OpenSMTPD) with ESMTPA id ba2e1b98; Sat, 24 Mar 2018 11:07:56 +0000 (UTC)
Date: Sat, 24 Mar 2018 11:07:56 +0000
From: Job Snijders <job@ntt.net>
To: dnsop <dnsop@ietf.org>
Message-ID: <20180324110756.GE69302@vurt.meerval.net>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
X-Clacks-Overhead: GNU Terry Pratchett
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: <https://mailarchive.ietf.org/arch/msg/dnsop/9f3W3YcmFLvXq4ZFOsteyCgxhHY>
Subject: [DNSOP] raising the bar: requiring implementations
X-BeenThere: dnsop@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: IETF DNSOP WG mailing list <dnsop.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/dnsop>, <mailto:dnsop-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/dnsop/>
List-Post: <mailto:dnsop@ietf.org>
List-Help: <mailto:dnsop-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/dnsop>, <mailto:dnsop-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 24 Mar 2018 11:08:12 -0000
Dear DNSOP, I'd like to share some pointers from the working group that governs the BGP protocol, IDR, on requiring implementations before drafts can advance towards RFC publication. Raising the bar for publication will take weight off the camel's back. The IDR policy and rationale is outlined here: https://trac.ietf.org/trac/idr/wiki An example of an implementation report is available here: https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-large-community%20implementations Every normative (RFC 2119 / RFC 8174) term should expand into a compliance test, and it is particularly good to document where implementations deviate from what is required or suggested in the proposed specification. The work listed on this wikipage was done _before_ the draft was submitted to the IESG, and we intentionally sought to create more than the minimum amount of implementations required. In the case of the BGP large communities project the working group was particularly lucky because Pier Carlo Chiodi created an open 'regression testing'-style environment into which BGP speakers could be plugged in to assess their compliance: https://github.com/pierky/bgp-large-communities-playground https://github.com/pierky/bgp-large-communities-playground/blob/master/tests/README.md The DNS world would benefit from such environments and automated compliance testing. Manual testing for a specification (that is still being worked on) can be tedious, having a 'playground' can pay huge dividend. We did catch bugs in implementations thanks to this environment, so it not only helped the specification, but increased the quality of the participating implementations. RFC 7942 suggest that During the development of a draft people are encouraged to document what implementations exist, an example is here: https://tools.ietf.org/html/draft-ietf-idr-large-community-12#section-7 Closed source implementations as part of such a list is not an issue (just a little bit more work), at IETF meetings we'd meet up, plug laptops with virtual machines into each other and run compliance tests. Sidenote: when IANA codepoints are needed but not yet assigned, don't forget to keep everything on separate beta/feature branches; don't ship software with squatted codepoints :-) The DNSOP working group will have to figure out what constitutes a meaningful implementation in what context. For example, a tcpdump or wireshark decoder would hardly count as an implementation of a proposed DNS feature, but those decoders are _incredibly_ useful to have alongside real implementations, especially during development. DNS people will probably want to see multiple implementation reports in context of packet decoding, stub, resolver, and authoritative. Kind regards, Job
- [DNSOP] raising the bar: requiring implementations Job Snijders
- Re: [DNSOP] raising the bar: requiring implementa… Petr Špaček
- Re: [DNSOP] raising the bar: requiring implementa… Willem Toorop
- Re: [DNSOP] raising the bar: requiring implementa… tjw ietf
- Re: [DNSOP] raising the bar: requiring implementa… Mukund Sivaraman
- Re: [DNSOP] raising the bar: requiring implementa… bert hubert
- Re: [DNSOP] raising the bar: requiring implementa… Matthijs Mekking
- Re: [DNSOP] raising the bar: requiring implementa… tjw ietf
- Re: [DNSOP] raising the bar: requiring implementa… Tony Finch
- Re: [DNSOP] raising the bar: requiring implementa… Job Snijders
- Re: [DNSOP] raising the bar: requiring implementa… Paul Vixie
- Re: [DNSOP] raising the bar: requiring implementa… Frederico A C Neves
- Re: [DNSOP] raising the bar: requiring implementa… Frederico A C Neves
- Re: [DNSOP] raising the bar: requiring implementa… Mukund Sivaraman
- Re: [DNSOP] raising the bar: requiring implementa… Mukund Sivaraman
- Re: [DNSOP] raising the bar: requiring implementa… Mukund Sivaraman
- Re: [DNSOP] raising the bar: requiring implementa… Arsen STASIC
- Re: [DNSOP] raising the bar: requiring implementa… Tony Finch
- Re: [DNSOP] raising the bar: requiring implementa… Tony Finch
- Re: [DNSOP] raising the bar: requiring implementa… Jan Komissar (jkomissa)
- Re: [DNSOP] raising the bar: requiring implementa… Phillip Hallam-Baker
- Re: [DNSOP] raising the bar: requiring implementa… Suzanne Woolf
- Re: [DNSOP] raising the bar: requiring implementa… Mukund Sivaraman
- Re: [DNSOP] raising the bar: requiring implementa… Phillip Hallam-Baker
- Re: [DNSOP] raising the bar: requiring implementa… Matthijs Mekking
- Re: [DNSOP] raising the bar: requiring implementa… Petr Špaček
- Re: [DNSOP] raising the bar: requiring implementa… Petr Špaček
- Re: [DNSOP] raising the bar: requiring implementa… 神明達哉
- Re: [DNSOP] raising the bar: requiring implementa… tjw ietf
- Re: [DNSOP] raising the bar: requiring implementa… 神明達哉
- Re: [DNSOP] raising the bar: requiring implementa… Andrew Sullivan
- Re: [DNSOP] raising the bar: requiring implementa… Evan Hunt
- Re: [DNSOP] raising the bar: requiring implementa… 神明達哉