Re: An IETF repository for working code in our protocols?

Vijay Gurbani <vijay.gurbani@gmail.com> Wed, 19 August 2020 20:23 UTC

Return-Path: <vijay.gurbani@gmail.com>
X-Original-To: wgchairs@ietfa.amsl.com
Delivered-To: wgchairs@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D3C623A0DF1 for <wgchairs@ietfa.amsl.com>; Wed, 19 Aug 2020 13:23:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id poiaO7UHq3WV for <wgchairs@ietfa.amsl.com>; Wed, 19 Aug 2020 13:23:10 -0700 (PDT)
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5A3CC3A0DEF for <wgchairs@ietf.org>; Wed, 19 Aug 2020 13:23:10 -0700 (PDT)
Received: by mail-ed1-x531.google.com with SMTP id m20so19185189eds.2 for <wgchairs@ietf.org>; Wed, 19 Aug 2020 13:23:10 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0wFJM3zXf4Y8+AKslDryImroADcZ7t2V1PcMTGOAXb8=; b=qZO2kYy9AdNk7TKXLPjFfVWr4WhqiP96TsLR8CqIaXt//wB0dVhYyXUBA15+aAHJ3x jgpHoSgZLQHSXXe4H419Y0iuSNB/NmyNUx42+doO94QKL4irUx2L+AuJtzaXb7qOVmV5 nlYuTcNgQAMtzBdgiMfBuJ0xv77dzsvucIE8VJD89jDxEWW2fjmFEcyW89uL3f99t6Gb pU60UChMVBVZtDNZK93hXjihVcH3L16e0qbj7MWCAMupIcsMwi1xV/y015jzPWo9OGpa a576ZQy3xcDcjSUv7Ez0DZu3HIuyWxH75WE8Et2mydbrfiPmLbka/Tsi+hQdr68WI90m pClw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0wFJM3zXf4Y8+AKslDryImroADcZ7t2V1PcMTGOAXb8=; b=sd0lChliPR+q86XsUqKWbiOztOtdOKwjCC3rRo0EXAwUfwIWXQdLp0PsKdkmdVJKAy 0sKvbNnKLlg+F3mSPofZWu3NhqhWYMw4/rcY0W4Ed/92SxgC6ub8cZih9edoPqFoI+5D Eco9y0qgQRjqxZQaMFVenozBTDTFS5vm8AwO+7myxQ+Vikd5uDtwQgfRB4IFUkFIckpD BDFb+9qHGgyoctV7JGC8ILKIxB/NHKAAoLd61fOszI4M9uxVBx8bQVdl9nObY6Xsb8GN Tc6EB0reP3AnuJcv/7A5DujBsjYYm6VUKzdTbSFB5GBjWGTrnegO6hZo4EwLYrK2z2RB 3g1Q==
X-Gm-Message-State: AOAM530DLGi1NZgZE+4xje89RCKf0HcMhVNLEIs2ixY4HHjUOun8BO7P sf+QhgI0DbquoxOGj222y4Y+YMptbpgg5W0h+voOYVMYfmGOdCBI
X-Google-Smtp-Source: ABdhPJzDYs+4qvnblzenj+zF8w1880ZIswJamOz+1SxmbEnSeGNSIVXsOC8eX7K1jNpQh0t4iPiUjQXHJ39l6QHccgA=
X-Received: by 2002:a50:cbc3:: with SMTP id l3mr26488757edi.58.1597868588795; Wed, 19 Aug 2020 13:23:08 -0700 (PDT)
MIME-Version: 1.0
References: <81300C20-AC38-465C-A17C-743F3D4CD947@nbcuni.com> <CAMMTW_+P60jB-MLsB6R_x7z3uk5xK56ZwkZnHOtzaxex3tDREA@mail.gmail.com> <CAMGpriX+auKUK7TrkLxd=Pdy_CH=s_X7BgYV24vO44ORAeAQnw@mail.gmail.com>
In-Reply-To: <CAMGpriX+auKUK7TrkLxd=Pdy_CH=s_X7BgYV24vO44ORAeAQnw@mail.gmail.com>
From: Vijay Gurbani <vijay.gurbani@gmail.com>
Date: Wed, 19 Aug 2020 15:21:39 -0500
Message-ID: <CAMMTW_KBLnv47GEGA-RvVxBbBcSepJFFn9gWduAnsPiS7UiBVQ@mail.gmail.com>
Subject: Re: An IETF repository for working code in our protocols?
To: Erik Kline <ek.ietf@gmail.com>
Cc: "Deen, Glenn (NBCUniversal)" <Glenn.Deen@nbcuni.com>, "wgchairs@ietf.org" <wgchairs@ietf.org>, Martin Duke <martin.h.duke@gmail.com>, Tommy Pauly <tpauly@apple.com>
Content-Type: multipart/alternative; boundary="000000000000a20c5705ad40c5f8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/wgchairs/zuYSjG9ePna8fkjIOGeDaRGA7D8>
X-BeenThere: wgchairs@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Working Group Chairs <wgchairs.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/wgchairs>, <mailto:wgchairs-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/wgchairs/>
List-Post: <mailto:wgchairs@ietf.org>
List-Help: <mailto:wgchairs-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/wgchairs>, <mailto:wgchairs-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Aug 2020 20:23:12 -0000

Dear Erik: Thank you for your thoughts and your time.  Please see inline.

On Wed, Aug 19, 2020 at 3:09 PM Erik Kline <ek.ietf@gmail.com> wrote:

> I tend to think that the value of example implementation code varies over
> time, in no small part because people often need to redo an implementation
> because of development language requirements, or they're building a
> component that subsumes/merges several protocols, etc.
>

Yes, absolutely.  The code archive should not require that the code be the
most optimized code with the smallest runtime and space complexity
measures.  And because the code will be archived upon at the end, when an
I-D becomes an RFC, hopefully it reflects the merges of several protocols.
(For those protocols where this is important, of course.  Not all protocols
are subsumes/merges of others.)

The things I find most useful are the those like the NIST and crfg RFC
> style of test vectors and expected outputs (and even sometimes expected
> internal state!).
>
> I would love to see some kind of standard approach for documenting what
> are essentially unit tests: given these inputs/this state/this HTTP method
> call/... -> expect these outputs/state changes/lists of observable behaviour
>

Those constructs you mention --- protocol style unit tests --- are
extremely valuable.  The SIP working group, when it was active, produced a
few such documents that contained complete protocol flows that aided in
testing and understanding of how the protocol works.  Many of these
documents were made into Informational RFCs.  The ALTO working group had
also produced a HTTP style RESTful API document that outlines the state of
HTTP headers and payload; alas, it did not progress to an RFC, but it was
very helpful when we had interoperability events for ALTO.


> Given the centrality of interoperability to the IETF, this seems like
> something we might be able to explore.
>

Yes, absolutely.  However, the working code (good working code) being
available is a bit of an orthogonal, though equally important, discussion
as is the one for establishing test vectors.  So at least I would not like
to conflate these two at the current time.

Thanks much!

- vijay