[SCITT] Re: Proposal: publish what the last week actually produced, as a citable interop report with all of us on it

[Anton Sokolov <[e-mail]>]

Joel,

Yes, and thank you for asking rather than assuming.

One scope note for my row, in the same spirit as Pablo's and Iman's, because
what I contributed is narrower than "ECDSA work" suggests, and I would
rather
fix that before a draft than after a DOI.

What is mine is one question about SCITT: from which bytes the identity of a
registered Signed Statement is derived, and what "the same statement" means
when the COSE_Sign1 encoding is not byte-unique. That question is upstream
of
your repair, which is how you already worded it, and the wording is right.

What is not mine is the underlying phenomenon. Signature malleability under
ECDSA is on the record: draft-prabel-cfrg-suf-hybrid-sigs Section 6.2.1
names
double receipts and log poisoning directly, and a JOSE/JWT treatment with a
library census was published in June. I made a claim on this list in August
that was wider than that, and I corrected it in the same thread. If the
report
says "found" where the right word is "raised for SCITT", it inherits that
overclaim from me, and I would rather it did not.

For the run-kind column: that week I contributed a finding rather than
running
anyone else's suite, and my row should say so plainly rather than sit beside
Walter's, Emek's, Tiago's and Henri's as though it were the same kind of
thing. Your own framing already separates the two groups; I just want my row
on the right side of it.

The rest of the shape is right. The non-comparable-pairs column is the part
I
would defend hardest if anyone argues it out: a report that records only the
comparisons that worked fails in the same way as one that records only
agreement, one level up.

Send the draft and I will read my own sentences before anything is
deposited.

Anton

Anton Sokolov
Tyche Institute MTÜ, Estonia
doctoral researcher, Tallinn University of Technology (from 1 September
2026)

On Tue, Aug 25, 2026 05:24 AM, Joel Hillier <[e-mail]> wrote:

> Hi all,
>
> Iman has put an open invitation on the list for a compatible
> implementation to run one COSE test set across two implementations before
> IETF 127, so the working group gets a concrete result rather than an
> assertion. That is the right instinct and it deserves more than one pair.
> Something happened on this list over the last week that I don't think any
> of us planned, and it's worth writing down properly before it scrolls away.
>
> *What actually happened, stated at the precision I can support.*
>
> Five of us ran code somebody else wrote, at pinned commits or package
> digests, on different operating systems and different language runtimes,
> and published the results so anyone can check them. Three more contributed
> findings without running anything, including one review frozen and
> OpenTimestamps-stamped before its author had read anyone else's. Along the
> way:
>
>    - Two verifiers written independently from one text disagreed on 15 of
>    15 documents, and the root cause was a member name the text never gave.
>    - A published schema was found to make a normative MUST unsatisfiable
>    at two of three sites where other rules rely on it.
>    - An anchored sequence was shown to be poisonable by anyone willing to
>    pay for one transaction, and its author's own anchor was then refused by
>    the rule that fixed it.
>    - A conformance protocol was shown to be unsatisfiable where its rules
>    entail one another, analytically by one person and empirically by another,
>    inside a vector its author had written.
>    - A single exit code was found to name five different conditions, and
>    the guard over the set of codes was blind to it by construction.
>    - A valid JSON file was found to be reported as invalid JSON, which is
>    a false message rather than a wrong code.
>    - A manifest runner was found to enumerate what was present on a build
>    machine while a constant at the top of it said tracked, so a file in no
>    commit would have been hashed into a published archive digest.
>    - A COSE_Sign1 envelope digest was found to be standing for two things
>    a relying party has to tell apart: the registration entry, and the
>    authorization claim inside it. That is Iman's, it's published as ten
>    source-locked cases, and it is the same shape as the exit code and the
>    verdict vocabulary above, in a different layer of the stack.
>
> *Three of those are already repaired, and that matters more than the list.*
> The exit-code and JSON-message findings are fixed in @conarium-ai/core
> 0.2.44 through 0.2.46, so exit 20 now names one condition rather than five
> and the JSONL message says what is actually true. The manifest runner is
> repaired in my tree, carries two mutants, and the tree it describes now
> verifies 30 of 30 with every enumerated file tracked by git. Anyone
> reproducing any of the originals needs the pinned version it was found at,
> which is precisely why the report has to record what was run against what
> rather than only what was found. A finding without its version is an
> anecdote a month later.
>
> Not one of them was found by reading.
>
> *And the last one generalises, which is the argument for doing this
> properly.* I measured signature stability across the algorithms a signing
> estate migrates through: one key, one signing input, 200 ordinary sign
> calls per leg, no adversary and no transform. The Sig_structure digest is 1
> of 200 on every leg. The envelope digest is 1 of 200 on Ed25519 alone, 200
> of 200 on ECDSA P-256, 200 of 200 on hedged ML-DSA-65, and 200 of 200 on a
> parallel Ed25519 plus ML-DSA-65 composition even though the classical leg
> alone is 1 of 200. Hedged and deterministic ML-DSA-65 are both FIPS 204,
> both verify, and are selected by a flag absent from the protected header,
> so two conforming signers disagree about whether the envelope digest is
> stable at all. The probe and its run record are pinned in my conformance
> tree and the numbers reproduce on two machines. Detail is in the agenda
> thread.
>
> *What I'd like to propose.* That we publish it as a joint interop report.
> Not a draft, not anybody's specification, and not an argument for anyone's
> document. A record of what was run, by whom, against what, and what it
> showed.
>
> *One.* A table of every run: implementation, author, what it was written
> from, what it was run against, the pinned commit or package digest, and the
> result. Using the run-kind vocabulary Iman wrote for his own row and Emek
> then applied to his: reproduction of author-supplied checkers, independent
> implementation from the text, or independent implementation with
> independent vectors. A row that doesn't name its kind reads as the first.
> Henri has since shipped that field in Vaara v1.75.0, so the vocabulary
> exists in a form other people can file against rather than only in this
> thread.
>
> *Two.* The disagreement matrix. Where two implementations differed, what
> they differed about, and which of them the text could actually settle. The
> disagreements are the load-bearing part. A report recording only agreement
> would be worthless and everyone here knows it.
>
> It also needs a column the last two days argued for: *what pairs of
> implementations are not comparable, and why.* Emek and Iman established
> on the list that two runners in different problem domains produce no
> cross-implementation result between them, and said so rather than letting
> the adjacency imply one. A report that records only the comparisons that
> worked would misrepresent the exercise as neatly as one that records only
> agreement.
>
> *Three.* The findings, each attributed to whoever found it, with the
> version it was found at and the version it was repaired in where it has
> been. Including the ones people found in their own work. There are a lot of
> those and they are the reason the exercise is credible at all.
>
> *Four.* A DOI, so it can be cited rather than linked. Henri has already
> deposited Vaara this way at 10.5281/zenodo.22029444, and it turns a
> repository URL into a reference. I'll do the deposit and the editing unless
> someone would rather.
>
> *Who's on it.* Everyone who ran something or contributed a finding, as an
> author rather than an acknowledgement. On the work so far that's Walter
> Hawkins, Emek Can Doğru, Tiago Pinto, Henri Sirkkavaara, Iman Schrock,
> Pablo, Nenad Vasic and me. Anton, your ECDSA work is upstream of a repair
> in my document and I'd put you on it too if you want to be. Todd, Walter
> named the receiver vantage as yours and it belongs in the report, but I'm
> not putting your name on something you haven't written a word of, so tell
> me either way. Iman, the consent you gave Anton covers naming EMILIA in the
> 127 discussion; an authored row here is a different ask, so tell me rather
> than have me assume it carries over.
>
> If I've mis-stated what anyone did, say so and it gets corrected before
> anything is deposited. I'll write the first draft and circulate it here.
> Nobody's name goes on it without them reading the text, and any of you can
> strike any sentence about your own work without giving a reason.
>
> *Why it's worth the effort.* Two reasons, and I'd rather state the second
> plainly than have it inferred.
>
> The first is that this list is about to have a conversation about what
> happens next, and the chairs have opened it: the IETF 127 agenda call went
> out with implementation and interoperability experience named as a
> category. The chartered work is close to done. RFC 9943 published in June,
> SCRAPI in the RFC Editor queue, and the CCF profile in Last Call. Meanwhile
> a substantial body of individual work has accumulated around this group and
> almost none of it is in the current charter. Off my own reference lists
> rather than a count I'd have to defend, that includes attested agent
> payment, disclosure evidence, canonical action identifiers, payload
> binding, agent accountability composition and conformance, agent records,
> web bot auth in three parts, receipt formats, and two of mine. Three more
> landed in nine days, two of them within 48 hours of my own last revision.
> When somebody asks whether there is work here to charter, the honest answer
> is either an opinion or a measurement, and right now we are the only people
> who can hand over a measurement.
>
> The second is that it is good for every one of us commercially, and
> pretending otherwise would be silly. I sell verification. Several of you
> sell adjacent things. A public, reproducible, adversarial interop study
> with eight names on it is worth more to all of us than any of us saying the
> same thing alone, and it is worth more precisely because none of us
> controls it.
>
> *What it is not.* It is not a vehicle for ARP or for CAP-1, and I'll take
> both out of the framing entirely if anyone thinks they're leaning on it. My
> drafts appear in it exactly as everyone else's do: as things that were run,
> with what broke.
>
> Say yes, no, or yes-with-changes. If the answer is no I'll drop it without
> argument, and the work stands on its own in the archive either way.
>
> Joel
>