[SCITT] Re: New draft: The Coverage Attestation Profile (draft-hillier-coverage-attestation-00), and a request to break it

[[e-mail]]

Joel,

Two answers, both measured. The second is worse for me than you put it.

The line endings: a checkout setting on my side, and your bundle is not wrong.

core.autocrlf=true is the Git for Windows installer default and it is set at system level on this machine, so every clone inherits it and nothing in a repository-local config reveals it. Your tree at 0980d32, same commit, same files, two checkout settings:

    draft-hillier-coverage-attestation-00.txt    32,544 bytes  ->  33,272

    cap-1/README.md                               4,128 bytes  ->   4,218

And the part that answers you directly: from a clean checkout the two digests Walter pinned reproduce over stored bytes with no normalisation at all.

    170aa81efc74c5278a2fb6e3bcc22bc91fb1706e9fb8faf1ce6d575e5ce3d965  cap-1/src/vectors/manifest.json

    4453f216089543780bfecc4295cc4a61462fdc585b88d1e35b7d1aba79716b4a  cap-1/src/CAP-1.schema.json

So my sentence was worse than imprecise. "Reproduce exactly once line endings are normalised" described a repair for damage my own checkout had done, and it was one step from becoming a normalisation step in somebody's construction. Please don't carry it anywhere.

The class digest: you are right, and the reason is worse than an ambiguous sentence.

There is no script. The value came from a computation I ran once and did not keep, so what I published was not a description of a construction; it was all that survived of one. Forty-four readings failing is the correct outcome. A forty-fifth that happened to land would have been worse, because then we would both have believed the sentence.

So I am not defending the number. I have taken your six points instead, because a list carrying two constructions is the thing we are both arguing against. Over the fifteen at 0980d32, enumerated from manifest.json rather than from a directory listing, stored bytes, no normalisation:

    ce88ddfc0feaec90d36fa81bd596ccdd674b349287b7ce24ed50f429c51cdfb7   15 members

One freedom your six points still leave, and it cost me three values to see it. They fix the bytes of each line but not what the path in the line is relative to. Three readings of that one degree of freedom:

    ce88ddfc0feaec90...   paths relative to the vectors directory  (the value above)

    f2096f3ab6c58e61...   paths relative to the repository root

    53d8d6ea24fd2423...   the first, with manifest.json as a sixteenth member

A seventh point closes it: the path is relative to a named root, and the root is published beside the digest. Mine is cap-1/src/vectors.

Your path-blind mutant, reproduced here rather than taken on trust. Over the same fifteen, swapping the contents of NC-01 and NC-02:

    path-bearing   ce88ddfc0feaec90...  ->  237e8eae3b6493f9...   sees it

    path-blind     2cd8ad89f5fa0125...  ->  2cd8ad89f5fa0125...   UNCHANGED

I got that wrong on the first attempt in a way worth writing down, because it is the same failure one level down. My first path-blind variant ordered the members by path and then dropped the paths, and it did see the swap, which made your mutant look wrong when it was my construction that was. A digest that commits to the multiset has to order the digests, not the paths. The ordering is part of what "over digests alone" means, and I had left it to the reader exactly as my original sentence had.

Emek

On Fri, Aug 21, 2026 at 5:07 PM Joel Hillier <jhillier=[e-mail]> wrote:

Hi Walter, Emek, Tiago, Iman,

Well, you all took me seriously when I said break it, and inside a day. Thank you. This is the loop working exactly as it should, and it's a far better use of a draft than agreement wouldhave been.

Emek has now run both his text-derived verifiers against the published fifteen, and that result is the news, so it goes first. Everything else is conceded below it, and the pinned bundleis at the end.

Two readings of my document, run against my own class, agree on nothing.

                 verdicts   strict attribution
    reader A       14/15          9/10
    reader B       10/15          0/10
    A vs B disagree on 15 of 15

Fifteen for fifteen. Not a divergence on an edge case, a total divergence, and it has one root: the draft doesn't name the members.

And the root under the root, which I would not have found by reading either. Emek's finding is that the published schema makes Section 9's MUST unsatisfiable. I checked it againstthe schema rather than take it on trust, and it reproduces exactly:

  subject.digest        object, required [algorithm, value]   states it
  catalogue_digest      string ^[0-9a-f]{32,128}$             cannot
  withheld_digest       string ^[0-9a-f]{32,128}$             cannot

Section 9 says a producer "SHOULD use an algorithm that is unbroken at time of production and MUST state which was used." The pattern spans 32 to 128 hex, which is MD5 through SHA-512, sothe algorithm isn't recoverable from the value. And both enclosing objects, basis and the unexamined item, are additionalProperties: false, so a producer who wants to comply has nowhere to put the statement. At two of the three sites where R3 and R4 rely on it, the schema forbids compliance with thedraft's own MUST.

That isn't a style inconsistency between two shapes. It's a normative requirement that cannot be met, and it's the root of the string-versus-object split Emek measured yesterday as eightdisagreements in twenty. The first shape is the one to keep. {algorithm, value} goes to the other two, because the string form is exactly what the weakest-rung argument is about: a digest whose algorithm is unstated is a claim whose strength thereader can't evaluate.

One correction to Emek's own finding, and it makes it worse rather than better. He writes that subject is an object with kind and ref, and that neither word occurs in the Internet-Draft. ref occurs zero times, which is right. kind occurs three times, and every one of them is basis.kind or the ordinary English word. So the schema requires subject.kind, and a reader who does meet the word meets it bound to a different object meaning something else. That's worse than absence, because absence at least reads as absence.

Reader B took R0's "MUST name a subject" at face value, expected a string, and refused all fifteen under shape, which is why its attribution is 0/10. Emek's line for it is the one I'd keep:a class where every document fails the shape check tells its author nothing about the rules underneath. And withheld_digest cost reader A its only lost verdict on PV-04, because it looked for digest and for hash, the two names the prose suggests, and found neither. One member name, and a conforming document reads as non-conforming.

NC-05, in my own negative control, without either of you arranging it. Reader A refuses NC-05 under R1 and R5 together. That's the analytic point arriving empirically inside the vectorI wrote: a document violating R5's second sentence has already violated R1, so the mutant survives R5's removal. Emek reached it by reading and Walter reproduced it from the other side. My own control shows it, which is better evidence than either readingof it was.

Walter found what sits underneath: whether 7.2's "the class MUST fail" incorporates 7.1's refused-by-the-rule-it-targets requirement. Attribution-aware, his R5 mutant dies. Mere-refusal,it survives, which is the survival Emek measured. Same protocol, two readings, opposite kill counts, and it's the load-bearing sentence of the whole conformance section.

Emek, on the part of your message that wasn't about my draft. You could have pointed only outward and you didn't. An implementer reached 13/13 against your published vectors onlyafter opening your source, and four things were nowhere in your documents: what the Ed25519 signature actually covers, that the sequence counter advances by exactly one, which absent fields are schema errors rather than tampering, and the order the checksrun in. The last one is the one worth the thread: run the signature check first and a deleted receipt reports as a bad signature. True verdict, wrong reason. Check order is a wire fact and almost nobody writes it down. That's the same failure as kind and ref, and you found yours the same week you found mine.

So the claims become, and all of it goes into -01 with mechanism rather than as quieter sentences:

Nine rules of which eight are exercised, R0 tested by nothing. Walter has that one concretely: silencing R1 through R8 kills the class eight times, silencing R0 leaves it passing, becauseno vector exercises shape alone, so a verifier with no shape checking whatsoever agrees on all fifteen. One R0-only negative control closes it, and until it exists the schema is normative and untested in the same breath.

Agreement among implementations on the rules and not on the normative shape. That sentence is now measured rather than hedged.

And the production-fixture claim comes out. Walter: Section 7.4's fixtures contain no cap/1 document at all, both are certisyn.verdict-document/1.0.0 hosts carrying the older coverage shape, and grep for cap/1 returns nothing in either. So "correctly refused under R1" has no referent in the artefacts it cites. Your line for it is right: the claims most likely to be false are the ones nothing was pointed at, and these were never pointed at a CAP-1verifier.

The denominator has two doors, not one.

Walter moves the edge after the answers are known: 500 checks, forty ugly, declare 460 on the declared basis, eight for eight, complete: true, perfectly accounted, about a population drawn after the fact. His sentence goes in the draft: the information didn't survive being moved from the remainder to the denominator, it just moved.

Emek then leaves the edge alone and moves the claim instead. A stratum states which classes of claim it supports and nothing requires the cited claim to fall inside what it states. FiveTLS configuration checks declaring supports: ["absence-of-any-vulnerability"] and bounding "no vulnerability was observed in the release" is conforming under both his verifiers. R8 checks presence. The failure it names is containment.

The R7 incentive is the third face. A check you expect to error is safer never dispatched, because running it and erroring costs complete while declining to run it keeps complete with a tidy disposition. It rewards not looking, and nobody has to write a false statement to use it.

The closed vocabulary isn't closed, and you found the boundary from four directions. not_sampled and not_yet_due from Walter, and his R4 argument settles the first: if an enumeration must state its method because the method determines what the enumeration could not see, then sampling is such a method and the undrawnunits are none of the eight. access_denied, because landing an active refusal beside a missing licence loses the most interesting event in the run. aborted_before_dispatch from Tiago, with the sharper half being that integrity.complete speaks only of dispatched units, so a clean operator cancellation keeps complete: true while part of the eligible population was never reached. And Emek's unit that was dispatched, ran, terminated normally and determined nothing, which failed misdescribes and unavailable misdescribes and examined counts anyway.

Emek, you declared an interest before anyone could notice it. That case is right on its merits and I'd have reached it more slowly without you naming it. superseded I want and am not adding yet, because the successor pointer is a different shape from the other dispositions and I'd rather get it right than get it quickly.

The basis trichotomy, and two repairs that aren't alternatives. Walter and Tiago reached the derived population independently, and Tiago's note is the one I'd keep: a composable or derived basis preserves how the population was obtained, Walter's enumeration digest fixes which population was obtained, and neither substitutes for the other because a digest of a set doesn't let anyone re-derive it without the source and the predicate. Both go in. Walter's re-derivability axis then replaces the trichotomy: external artefact,internal artefact, or none. That partition doesn't leak, and declared comes to mean exactly one thing. The argument that convinced me is that the careful ones currently get punished, and a vocabulary that penalises rigour gets routed around bythe people you most want using it.

withheld is incoherent, and the vectors already disagree with the text. Tiago has it analytically: Terminology says a disposition is why a unit was not examined,Section 6 says withheld is examined, Section 4.1 puts it in unexamined, and R1 counts. No count assignment preserves all four. Plus R2 requires every unexamined entry to name a unit while the privacy rationale exists so a sensitiveunit needn't be named. Walter settled it empirically in both halves: PV-04 carries eligible 7, examined 4, unexamined 3 with the withheld unit among the three, so the class counts withheld as unexamined against Section 6, and the withheld entry names its unit, selector.ts, against the privacy rationale. The vectors have been practising the repair on one axis and refusing it on the other. Tiago's two-axis split, examined and disclosed, is the fix.

---

The pinned bundle. Built, not promised.

class digest    742509bba744a1b6bf57a057a7769db00d583dbf656648218583a53dcac51254
bundle digest   7e3e54963de75316e41a957447b290739bfac5c408f9d973a66f04cf7313dca7
MANIFEST.json   0637ea1db3d330aa616e20c1727a191f1065b250e8f2f2df59db2345fe32ce59
archive         680e7d20862fee173c568cdf2f384e5bfa0d4fb07df614fc48b22f438fd241ad
members         42, of which 15 are the conformance class
source          Certisyn-Inc/certisyn-drafts @ 0980d3201aa2caab3cbad5c6e9bc99b422370b43

Walter, both digests you pinned reproduce exactly against that tree, 170aa81e... for the vector manifest and 4453f216... for the schema.

Emek, one question about your reproduction, and I think it's a real hazard rather than a nitpick. You wrote that Walter's digests reproduce "exactly once line endings are normalised."The bundle pins SHA-256 over stored bytes with no normalisation, deliberately, so if your checkout is translating line endings then the bundle digest will not reproduce for you as specified, and the artefact built to end this class of ambiguity has one init. Can you tell me whether that was a checkout setting on your side or a genuine difference in the stored bytes? If it's the second, the bundle is wrong and I'd rather know now.

Your class digest doesn't reproduce, and that turned out to be the most useful thing in the exercise. "SHA-256 over the sorted per-file sha256 list of the fifteen" fixes the informationand not the representation. I tried forty-four readings of that sentence against your value and none reproduced it: sorted by path or by digest, joined by newline or nothing or comma or space, with and without a trailing byte, hex or raw octets, with and withoutthe filename, in sha256sum order or reversed, over fifteen files or sixteen, over stored bytes or canonicalised JSON.

That isn't a criticism of the proposal. It's the proposal's own argument arriving one level up, and it's why the bundle specifies the construction to the octet instead of describing it.Six points, each naming a byte: enumerated member set never inferred from a directory listing; SHA-256 over stored bytes with no normalisation; ordered bytewise over the UTF-8 path; one line per member as lowercase hex, one U+0020, POSIX relative path; joinedwith U+000A including after the last; SHA-256 over that sequence.

One departure from your construction, with the mutant that earns it. The digest is over path-and-digest lines rather than over digests alone, because a list of content digests commitsto the multiset of contents and not to which file holds which. The bundle's mutation test swaps the contents of NC-01 and NC-02 and prints the path-blind digest either side:

path-blind class digest, before: 47314682fddd39da
path-blind class digest, after : 47314682fddd39da
UNCHANGED, so a digest over content alone cannot see this

The bundle carries its own verifier and its own mutants. verify_bundle.py, 42 members, 0 failures, and it reports every failure rather than stopping at the first, which is Emek's short-circuiting finding applied to the tool. That finding deserves its own line, because it generalises past both our drafts: refused by the rule it targets is only checkable against a verifier that reports every refusal. Against a short-circuiting one it's a fact about check order.

mutate_bundle.py, four mutants, four killed, and the crediting rule there is a set rather than a singleton, for your reason rather than mine. Altering a vector must move its member digest, the class digest and the bundle digest together, so a mutant cannot produce exactly one failure. That is NC-05 again, in a different tool, withinan hour of reading your report on it: a protocol demanding exactly one broken rule per control cannot be met where the rules entail one another. I hit it building the thing that exists because you found it.

Standard library, no network, no arguments. Tiago, that should meet your bar; if it doesn't, say where and I'll fix the bundle rather than argue.

Tiago's freeze discipline, agreed and extended: you, Walter and Emek's two readers each hash implementation and interpretation notes before seeing another's, then run against this. Agreementis then worth something and disagreement is worth more.

Iman, thank you for the priority note, and it wasn't necessary from my side. An additional implementation is exactly as useful as a fourth one, and PR #630 written from -00 is another reading of the same text by someone I haven't spoken to. Yes to the schema-shape and examined-set negative controls against the pinned commit, and they can now cite a class digest rather than a commit.

---

One thing your break has already changed elsewhere. I read Walter's first message, went looking for the same shape one level up in the other draft, and it was there. ARP's evaluationsweeps carry an accounting identity over a bracket of ledger sequence numbers and check that nothing was skipped inside the bracket. Nothing required the bracket to abut the previous one, so a sweep could be complete over an interval beginning after the entriesan operator would rather not have examined, and every rule passed. Now fixed: brackets must abut, and any range deliberately not examined is individually disposed with a registered reason, where the registry refuses a reason the reader can't check for itself.Walter's sentence is quoted there with his name on it.

Different subject matter, same defect, found by reading a review of a different document. Which is the strongest argument any of us has made for the shared substrate, and it wasn't madein the abstract.

Keep going. This is the most useful day either draft will ever get.

Joel