Return-Path: X-Original-To: scitt@mail2.ietf.org Delivered-To: scitt@mail2.ietf.org Received: from localhost (localhost [127.0.0.1]) by mail2.ietf.org (Postfix) with ESMTP id DCE2C12D6E1B4 for ; Fri, 21 Aug 2026 09:13:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ietf.org; s=ietf1; t=1787328816; bh=LER8TbOd9olfamAuclMLjqgkVyclG0Tgl2o1x3FTLmk=; h=Cc:From:In-Reply-To:References:Subject:To:Date; b=ZxO7zmJsFTbt1ptoB80B0av9F4mFkLdBArHw8KOQgXryl6kcmEj9jB9rxV8gYlghF tq359GSL9ZdG4C9Q3BNLYmvUzNTC0lKCr+bEbGvz98owDM2iYDIbd8ZpebPuZ2yNpj wx7uFOX5tISkv34X1vrEWItV7HtEmPMYt/iK5zMY= X-Virus-Scanned: amavisd-new at ietf.org X-Spam-Flag: NO X-Spam-Score: -1.618 X-Spam-Level: X-Spam-Status: No, score=-1.618 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, HTML_MIME_NO_HTML_TAG=0.377, MIME_HTML_ONLY=0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Authentication-Results: mail2.ietf.org (amavisd-new); dkim=pass (2048-bit key) header.d=conarium.dev Received: from mail2.ietf.org ([166.84.6.31]) by localhost (mail2.ietf.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2J69y1F-9uqS for ; Fri, 21 Aug 2026 09:13:36 -0700 (PDT) Received: from bonobo.yew.relay.mailchannels.net (bonobo.yew.relay.mailchannels.net [23.83.220.22]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by mail2.ietf.org (Postfix) with ESMTPS id 142F912D6E1B1 for ; Fri, 21 Aug 2026 09:13:35 -0700 (PDT) X-Sender-Id: hostingeremail|x-authuser|e.dogru@conarium.dev Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id D6A39942DDF for ; Fri, 21 Aug 2026 16:13:28 +0000 (UTC) Received: from fr-int-smtpout30.hostinger.io (100-117-166-221.trex-nlb.outbound.svc.cluster.local [100.117.166.221]) (Authenticated sender: hostingeremail) by relay.mailchannels.net (Postfix) with ESMTPA id 706659422E4 for ; Fri, 21 Aug 2026 16:13:28 +0000 (UTC) X-Sender-Id: hostingeremail|x-authuser|e.dogru@conarium.dev X-MC-Relay: Good X-MailChannels-SenderId: hostingeremail|x-authuser|e.dogru@conarium.dev X-MailChannels-Auth-Id: hostingeremail X-Spot-Plucky: 1107d44b51c713c0_1787328808813_1904334150 X-MC-Loop-Signature: 1787328808813:3267054078 X-MC-Ingress-Time: 1787328808813 Received: from fr-int-smtpout30.hostinger.io (fr-int-smtpout30.hostinger.io [148.222.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.117.166.221 (trex/8.0.2); Fri, 21 Aug 2026 16:13:28 +0000 Received: from localhost (34.86.89.34.bc.googleusercontent.com [IPv6:2a00:1d35:17a4:d900:f9a7:4dfc:5b46:5980]) (Authenticated sender: e.dogru@conarium.dev) by smtp.hostinger.com (smtp.hostinger.com) with ESMTPSA id 4hRQMp3Rl3z30xr for ; Fri, 21 Aug 2026 16:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=conarium.dev; s=hostingermail-a; t=1787328806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rL2Lq9tBaCEV+aIxprN+o0tf0Lh5Sxwd5HXFfzgagZY=; b=G/BymgsC0NwT15na6S2/oIGRJWMNm7dC+V9ttDKBKzjSk3P25DVEdYCVnUtfzUDP4JRwLw 9k9WJiOUTuo6aAo9f766nVzzntClgjkXKhGq/j7nILW9IDsggoCnqnWuOonZkNu0iSzsTe zUZmUwZT/x79cW+WSmhvUEjBTOvSR93TACgwdwN6O5pMeUiEGrZtZ8LNXX3k6+2nxytrba VLhPxyiwKUod/A0QgvJMXV6UTqr6S/bSI0ZhV8psQLtToAdGPTWYwOZ4JGu4NntSfir/bx ixmYDnQ3AqTg6AVzZfZLdOW+Qwr2GhT44j87IXJ98w8XfsnIegvZwMBs6DBTdw== Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 From: In-Reply-To: Message-Id: <1787328798802386477.1787328798@conarium.dev> Mime-Version: 1.0 References: To: Date: Fri, 21 Aug 2026 16:13:26 +0000 (UTC) X-CM-Analysis: v=2.4 cv=GMJaEfNK c=1 sm=1 tr=0 ts=6a887926 a=/mYa/mVueCak43dcYQE4Uw==:617 a=1Q4z3441aVnbLEOd:21 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=48vgC7mUAAAA:8 a=PzIs_5y6PMRoO2IC7OAA:9 a=EtRwWFWS7_2YulmF:21 a=frz4AuCg-hUA:10 a=QEXdDO2ut3YA:10 X-CM-Envelope: MS4xfHpay07szOnfzY3hxeZGUdr/jG3CrBtQRz8/2GsJEKvTHCpxu9SXOJXEXAsqtUKZc0RL0wTsVZZP/QDLusnfNA+0Q6FmRZJOBJd3T3Btyt8LB4E9GK6/ qLERdM9ygA5TvN9ApoU04VTY8fRtGtTSggd6BCBbiBwH0aU0ioaxw/zpxNXlR7enyK6CG0T/kOI/Yztzaj15KoroJQVBBbBMd66wMssfm6TiSYzq8WZcUIjf HPmXhlBzoPF71zE6AsdT0g== X-AuthUser: e.dogru@conarium.dev Message-ID-Hash: GWAPI6NCMAZJBOKSECXA5LOGTJMJWIMJ X-Message-ID-Hash: GWAPI6NCMAZJBOKSECXA5LOGTJMJWIMJ X-MailFrom: e.dogru@conarium.dev X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: scitt@ietf.org, team@emiliaprotocol.ai, hello@vaara.io, wdhawkins46@gmail.com, tiago@donttrustverify.pt, anton.sokolov@tyche.institute X-Mailman-Version: 3.3.9rc6 Precedence: list Subject: =?utf-8?q?=5BSCITT=5D_Re=3A_A_page_for_independent_runs_of_the_Vaara_conform?= =?utf-8?q?ance_vectors?= List-Id: "Supply Chain Integrity, Transparency, and Trust" Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe:
Joel, Henri,

Two applications of the kind fie= ld, one of them backwards at myself.

The row I fil= ed this afternoon (vaaraio/vaara#612) already names its kind, because
=
Iman's scoping was in front of me when I wrote it: a reproduction of t= he
author-supplied checkers and vectors at commit a209864e, not a= n independent
implementation of the receipt draft and not a valid= ation of the specification
text. So the field is fillable and tha= t is what a filled one looks like. Henri, if
you add it, my row m= aps to your first kind and needs no amendment from me.

=
Backwards at myself, twice.

First, the ARP ru= n I reported earlier is the same kind and I did not say so. Your
= runners over your vectors, on my machine =E2=80=94 it establishes that the = archive runs
and is byte-stable somewhere other than yours, and n= othing about the ARP text,
because I did not implement anything f= rom it. By your own rule, a report that does
not name its kind re= ads as the strongest one available, and mine did not name it.
Second, and this is the worse one: I reported six runners and s= aid the bundle was
run. Six is what your message listed. REPRODUC= E.md documents more than six that
need no arguments, and I took y= our count instead of deriving it =E2=80=94 on the same day
you co= rrected yourself for exactly that, in a message I had read. I have since ru= n
the two I missed:

    check_= reproduce exit 0
    mutate_reproduce exit 0 = 5 mutants, 5 killed on their designed set

Both p= ass. The coverage claim in my earlier message was six of eight, stated as
though it were all of them.

And running t= hem surfaced something in the archive. REPRODUCE.md step (k) is:
=
    # (k) check this file against the tree it desc= ribes. Two digests here
    # were stale before thi= s existed, and nothing caught them.
    python3 runners= /verify_manifest.py

runners/verify_manifest.py is = not in the self-contained archive. So the check
written because t= wo digests in that document went stale, with nothing to catch
the= m, is the one missing from the tree it shipped in. A reader following
=
REPRODUCE.md in order hits it at step (k) and finds nothing there =E2= =80=94 and the
failure mode is the one the step exists to prevent= , since what they cannot run is
the thing that would have told th= em the document's own digests are current.

I do no= t know whether it is excluded deliberately as part of the subset or left
out by accident. Either way the invocation is still in the file a r= ecipient is
told to follow.

The CAP-1 ru= ns are the one place I would claim the second kind, and only because
<= div>the two readers were written from the -00 prose before the vectors were= locatable
=E2=80=94 which is what the digests in the other threa= d now pin, whatever they are worth
about the past.

=
One thing your taxonomy does that I would keep. It makes what a = run establishes a
property of who wrote the verifier and what the= y read, rather than of the result.
Two green runs can be worth en= tirely different amounts, and until now nothing in
any of our reg= isters could say which was which. A class digest fixes the bytes; it
<= div>was never going to fix that.

Emek

On = Fri, Aug 21, 2026 at 6:30 PM Joel Hillier <jhillier=3D40certisyn.com@dma= rc.ietf.org> wrote:
=20

Hi Iman, Henri,

Iman, the last sentence = of that is the most useful thing anyone has put on the runs page, and I don= 't think it should sit inside one message.

You wrote y= our own row down before anyone asked you to: a reproduction of the author-s= upplied checkers and vectors, not an independent implementation or validati= on of the specification text. Nobody made you add that. It's a passing resu= lt you had every incentiveto file as a plain 41 of 44, and you named the gr= ound it stands on instead.

That's the weakest-rung = rule arriving at a register entry. We have now watched it turn up in bo= unds, in populations, in verdicts, in exit codes, in a freshness floor, and= yesterday inside a statistical estimator. Every one of those was a rule ab= out anartefact. Yours is the first at the point where a result gets file= d, which is the point where the loss actually happens, because a row ou= tlives the message that explains it.

So the specif= ic suggestion, and it's smaller than it sounds: the row carries what kind o= f run it was. Three kinds cover what this list has produced in a week, = and they aren't degrees of the same thing:

A reproduct= ion of the author's checkers over the author's vectors. Establishes that th= e artefact runs and is byte-stable somewhere other than the author's machin= e, which is worth having and is not nothing.

An indepe= ndent implementation from the text, run against the author's vectors. Estab= lishes something about the text, because a second reader had to decide what= the sentences meant.

An independent implementation ru= n against independently constructed vectors. Establishes something about bo= th.

And the rule that makes it load-bearing rather tha= n decorative: a row that doesn't name its kind reads as the first one. Henr= i, that's the same argument you already accepted for disagreement rows. A r= egister that can only record agreement is a marketing surface.A register th= at can't distinguish a reproduction from an independent implementation will= get cited for the second when it holds the first, and the person citing it= won't be lying.

Applied to what I sent this list a= n hour ago, before anybody files anything. The five runners in that arc= hive are my checkers over my vectors. If Henri runs them and gets my two pi= nned digests, that is a reproduction row and nothing stronger. It showsthe = package works outside the tree it was built in, which is the thing the 11 A= ugust package failed, and it shows nothing whatever about whether ARP's tex= t says what I think it says. I'd rather write that down now than have it in= ferred from a green row later.

And it's already loa= d-bearing on the other draft. CAP-1 has picked up runs of three differe= nt kinds this week and the numbers don't distinguish them. Walter's is an i= ndependent implementation from the prose plus the published schema. Emek's = two are independentimplementations from the prose alone, and one of them re= fused all fifteen documents on shape because the member names aren't in the= draft at all. Yours is a reproduction. Those are three different pieces of= evidence, and "three implementations agree" flattensall three into a sente= nce that was already the weakest claim in the document.

The class digest fixes which bytes were run. It records nothing about who= wrote the verifier or what they read to write it, and that's the axis that= decides what a run is worth.

Henri, if the row format= takes one more field, I'd make it that one rather than anything I suggeste= d earlier.

Best,

Joel


____________= ____________________________
From: Iman Schrock <team@emiliaprotocol= .ai>
Sent: Friday, 21 August 2026 15:00:19
To: hello@vaara.io Cc: scitt@ietf.org; Joel Hillier
Subject: Re: [SCITT] Re: A page for = independent runs of the Vaara conformance vectors

Henri,

= Thank you for taking this seriously and fixing both the claim and the mecha= nism.

The corrected language is accurate. The retained-head checks= and the
external witness close the issue I raised without overstating = what the
hash chain alone provides.

You may file my run exactl= y as scoped:

41 passed, 0 failed, 3 skipped across 44 suites at co= mmit
62a7080b7c854173c7d6b8ee51ce4dd724d59227. This is a reproduction o= f
the author-supplied checkers and vectors, not an independent
impl= ementation or validation of the specification text.

Best,
Iman=