Re: An IETF repository for working code in our protocols?

Erik Kline <ek.ietf@gmail.com> Wed, 19 August 2020 20:52 UTC

Return-Path: <ek.ietf@gmail.com>
X-Original-To: wgchairs@ietfa.amsl.com
Delivered-To: wgchairs@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B84A23A0E21 for <wgchairs@ietfa.amsl.com>; Wed, 19 Aug 2020 13:52:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QyJ362N5Xrfj for <wgchairs@ietfa.amsl.com>; Wed, 19 Aug 2020 13:52:44 -0700 (PDT)
Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2FE983A0E1E for <wgchairs@ietf.org>; Wed, 19 Aug 2020 13:52:44 -0700 (PDT)
Received: by mail-oi1-x235.google.com with SMTP id a24so115501oia.6 for <wgchairs@ietf.org>; Wed, 19 Aug 2020 13:52:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=byL85GoOy0Qu1awbq2F6RztPUsM+QYwzPfYqWl3pyRg=; b=KA3rHDaskMke6qF0N8dAGiPFnH4T8hgWfA/6L4dayreAZFyRykaXIiV2DYQYDnGMKh wH0MzI7ZaSvN8XrEWsco/4D557izOfzPLFKk69vmleY8CRo4QKt0Ulh6CcRuBWtvJ+yc 10TIvWSiCQxyuN/SWKGkOjF4cuxklFtsxF+lE2NPHh2IKPT4uA4yozZ7jms/eqYaFxYA 4aLOO2ZEjlVlCMlKT6MKIvIer6MsosewU5026Oi6nhXoGWUQ7U+wikMVacfLzUdqdPQq UyZIqTxAueBkW8fQaMTVpsp8p28MBVu0Dt86jlKqcfQUjLolDe7lJf1gWuxGLOZ1+xt1 B/hw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=byL85GoOy0Qu1awbq2F6RztPUsM+QYwzPfYqWl3pyRg=; b=Koe0GYlaKgMkTI5bNM0sZivu8jgErodjRk63zpJ2U1p5slBR/ck0lA8xQ90wZoUi1f 9VKYz5Ili8MiXhwsSB4txkrcrAuQZfiza41JDUFLcT5foe4xulxWMg59Lc4seg1FNh4c ovAmIOsD3BQmMS2w22gFexGxnmYr8diINNzblJcJa4bmWTRi+uUe7ZFDSNHygXQH+IBz jacTMO3Mn3AVsJKBzUVY9WsIXzyJeKLlOryyEXbnGWGDqqf7BszmcG2OSni8tZUIF/T4 OCJ8KJUYBPPwYYY/W+8fYq1Ic81GHUw3NrNVZuLkIpaJSSOWoafzgdHnZZmgfc66eu31 h8CQ==
X-Gm-Message-State: AOAM532tTO2di0AurFUIhmN0nc1FlTRyhSX1AYCxiCHgCi3OPwdrbvWy BRJVj8wOzL2mEBPKtGKfcs4Cum5SqtQYWrITgeU=
X-Google-Smtp-Source: ABdhPJxI8XzZMbjhjqlpXyJ/3GUtN9fia1MBLjK7L+kPhbn4LRBkVODbU/pDCXSa0uOJo7bAplM4m+R8wiBdrTmf+Iw=
X-Received: by 2002:aca:1117:: with SMTP id 23mr20268oir.97.1597870363443; Wed, 19 Aug 2020 13:52:43 -0700 (PDT)
MIME-Version: 1.0
References: <81300C20-AC38-465C-A17C-743F3D4CD947@nbcuni.com> <CAMMTW_+P60jB-MLsB6R_x7z3uk5xK56ZwkZnHOtzaxex3tDREA@mail.gmail.com> <CAMGpriX+auKUK7TrkLxd=Pdy_CH=s_X7BgYV24vO44ORAeAQnw@mail.gmail.com> <CAMMTW_KBLnv47GEGA-RvVxBbBcSepJFFn9gWduAnsPiS7UiBVQ@mail.gmail.com>
In-Reply-To: <CAMMTW_KBLnv47GEGA-RvVxBbBcSepJFFn9gWduAnsPiS7UiBVQ@mail.gmail.com>
From: Erik Kline <ek.ietf@gmail.com>
Date: Wed, 19 Aug 2020 13:52:32 -0700
Message-ID: <CAMGpriV-5pBC7znut8LzjYx29dyQiTg1yve-wbVHhbmh-EOAeA@mail.gmail.com>
Subject: Re: An IETF repository for working code in our protocols?
To: Vijay Gurbani <vijay.gurbani@gmail.com>
Cc: "Deen, Glenn (NBCUniversal)" <Glenn.Deen@nbcuni.com>, "wgchairs@ietf.org" <wgchairs@ietf.org>, Martin Duke <martin.h.duke@gmail.com>, Tommy Pauly <tpauly@apple.com>
Content-Type: multipart/alternative; boundary="00000000000069055a05ad412f42"
Archived-At: <https://mailarchive.ietf.org/arch/msg/wgchairs/ib3w6DkdCZQcjPmr4PA_hudhhxI>
X-BeenThere: wgchairs@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Working Group Chairs <wgchairs.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/wgchairs>, <mailto:wgchairs-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/wgchairs/>
List-Post: <mailto:wgchairs@ietf.org>
List-Help: <mailto:wgchairs-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/wgchairs>, <mailto:wgchairs-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 19 Aug 2020 20:52:46 -0000

>
>
> The things I find most useful are the those like the NIST and crfg RFC
>> style of test vectors and expected outputs (and even sometimes expected
>> internal state!).
>>
>> I would love to see some kind of standard approach for documenting what
>> are essentially unit tests: given these inputs/this state/this HTTP method
>> call/... -> expect these outputs/state changes/lists of observable behaviour
>>
>
> Those constructs you mention --- protocol style unit tests --- are
> extremely valuable.  The SIP working group, when it was active, produced a
> few such documents that contained complete protocol flows that aided in
> testing and understanding of how the protocol works.  Many of these
> documents were made into Informational RFCs.  The ALTO working group had
> also produced a HTTP style RESTful API document that outlines the state of
> HTTP headers and payload; alas, it did not progress to an RFC, but it was
> very helpful when we had interoperability events for ALTO.
>

Brilliant!  Yes, this was exactly the kind of thing of which I was thinking.


>
>> Given the centrality of interoperability to the IETF, this seems like
>> something we might be able to explore.
>>
>
> Yes, absolutely.  However, the working code (good working code) being
> available is a bit of an orthogonal, though equally important, discussion
> as is the one for establishing test vectors.  So at least I would not like
> to conflate these two at the current time.
>
>
But surely we'd want the code to path the protocol unit tests, yes?  ;-)