[SCITT] Re: New draft: The Coverage Attestation Profile (draft-hillier-coverage-attestation-00), and a request to break it

[Anton Sokolov <[e-mail]>]

Joel, all,

Late to this, so I read the thread through before writing and dropped the two
things I had that Emek, Walter and Tiago already have. What is left is one
dimension I could not find anywhere in the thread, and you have already
accepted a version of it one level up.

In ARP, a sweep could be complete over a bracket of ledger sequence numbers
that began after the entries an operator would rather not have examined,
because nothing required a bracket to abut the previous one. You repaired it
by requiring abutment and by disposing any deliberately unexamined range
individually, with a reason the reader can check.

CAP-1 has the same exposure in a weaker form, because a stratum has no bracket
at all.

Coverage in CAP-1 is not dated. as_of appears once, in the list of optional
top-level members in Section 4, and no normative rule refers to it. R4 makes
the denominator's provenance checkable — which catalogue revision, which
enumeration method, or the admission that it was merely declared. Nothing
makes its currency checkable.

Two consequences, and the second is the one I would fix first.

A catalogue basis pins which revision was counted against. It does not pin
when the counting happened relative to that revision. Take two documents
identical byte for byte, both citing catalogue digest D: one produced while D
was in force, one produced a year after the catalogue moved on. Both conform.
No verifier separates them, and the two readers are entitled to different
conclusions. That is the failure Section 1.1 describes, reappearing in the
denominator rather than in the remainder.

The second is inside the vocabulary. Several dispositions are facts about a
moment recorded as facts about a unit. unavailable is defined as "could not
be dispatched at all, for want of a dependency, a licence or a network". A
licence lapses on a date; a network is out for an hour; a dependency is absent
until someone installs it. disabled_by_policy is true of a policy version,
out_of_scope of an authorisation, and authorisations are amended. A CAP-1
document states which unit and which disposition, and not as of when — so a
reader cannot tell a standing property of the unit from the state of the world
on one afternoon. Re-run tomorrow and the same unit is examined, with no
contradiction to detect.

So the profile refuses a remainder that reconciles only by subtraction, and
accepts a denominator that reconciles only by an unstated moment.

That this is not a corner case, I can put a number on. On the EU Trusted Lists
— a catalogue of record in exactly R4's sense, versioned and published — 2,071
of 4,815 trust-service entries, 43 per cent, carry a status transition from
valid to revoked somewhere in their published history. All 26 reachable
member-state lists are affected, at rates from 6 to 81 per cent. Corpus, tool
and per-list provenance hashes:

  https://github.com/tyche-institute/time-travel-evidence-lab
  https://doi.org/10.5281/zenodo.20840168

Those are trust services rather than detector routines and I am not claiming
the populations are alike. The transferable part is narrow: a denominator
drawn from a versioned catalogue of record is not stable, so "counted against
revision D" and "counted at time T" are two different facts, and CAP-1 records
only the first.

The minimal repair, in the draft's own idiom and in the shape you already used
for ARP:

  R9, coverage is dated. Every stratum MUST state the interval over which
  its units were examined. Where basis.kind is catalogue, the cited digest
  MUST be the revision in force throughout that interval; where it was not,
  the stratum MUST be split at the revision boundary.

And then either give as_of normative force or delete it. An optional member
that no rule reads is somewhere for a reader to find comfort the verifier
never checked, which is the failure the draft exists to prevent.

Second, much smaller, and it is Tiago's rather than mine — he listed duplicate
unit identifiers among his smaller findings without developing it. The only
uniqueness requirement in -00 is in R0, on stratum identifiers. Nothing
requires the units inside unexamined to be distinct, so R1 can be satisfied by
listing one unit twice, which leaves one genuinely unaccounted unit outside
the accounting while the arithmetic closes. R1 exists to stop a remainder that
reconciles by subtraction; repetition reconciles it by addition and arrives at
the same place. "Individually accounted" gestures at distinctness, but a
verifier author has to infer it, and with four implementations in this thread
that is exactly the kind of inference two of them will make differently. One
clause in R1 or R2 settles it.

I have not written a verifier against -00, so treat the first item as a
reading of the text and not as a run. If it survives the thread I am glad to
write the interval half up as text for -01.

Anton Sokolov
Tyche Institute MTÜ, Estonia
doctoral researcher, Tallinn University of Technology (from 1 September 2026)

On Mon, Aug 24, 2026 at 10:01 PM <[e-mail]> wrote:
>
> Joel,
>
> Your question, answered by measurement rather than recollection: it was a checkout setting on my side. The stored bytes are fine and the bundle is not wrong.
>
> The setting is core.autocrlf. On this machine it is true at the system level, which is where Git for Windows puts it; I had not set it anywhere myself. With it on, every one of the forty files under cap-1/src differs on disk from its stored blob. NC-01.json is the smallest example:
>
>   stored blob 1303 bytes 31e430d5f67447fe3e3e09c154ecd29c396615f58bfb761b36f1d1a03189ebbf
>   working tree 1360 bytes 82e280734824963097c4edb310b6b5cf1e1d4638d81a86b98a4231b5b3cc23d5
>
> Fifty-seven bytes for fifty-seven lines, one CR each. Cloning the same commit with core.autocrlf=false gives a working tree where all forty files match their blobs exactly.
>
> With that checkout, your class digest reproduces here:
>
>   742509bba744a1b6bf57a057a7769db00d583dbf656648218583a53dcac51254
>
> Fifteen members, manifest.json excluded, paths relative to cap-1/, which is the root Tiago had to use as well. So his reading and mine now agree on the value and on why the other value existed: the construction fixes the bytes and the ordering, but not the reference point of "relative path" - and it does not fix the checkout that produces those bytes in the first place.
>
> The second one is worth a line in the bundle, because it is the same defect one level down. A .gitattributes in the repository, even just `* -text`, would make the stored bytes survive a clone on Windows, so a reader on that platform hashes what you hashed rather than what their Git wrote for them. There is none in the tree at 0980d32. Without it, the artefact built to end this class of ambiguity has a platform-shaped hole in it, and a reader who falls in cannot tell that from a real disagreement. I fell in, and reported it as one.
>
> The bundle digest I still cannot check, and my count of the tree agrees with Tiago's: at 0980d32 there are 52 tracked files, 49 under cap-1/ and 40 under cap-1/src. Neither verify_bundle.py nor mutate_bundle.py is among them. So I cannot derive the forty-two-member set from the published bytes either, and I would rather say that than guess at a composition.
>
> Emek
>
>
> On Mon, Aug 24, 2026 at 8:45 PM Tiago Pinto <tiago=[e-mail]> wrote:
>
> Joel,
>
> You asked whether the bundle clears my reproducibility bar. I wanted to
> answer that by actually rebuilding it rather than by reading the
> description and guessing.
>
> The class digest does reproduce.
>
> >From the public tree at commit 0980d32, I get:
>
> 742509bba744a1b6bf57a057a7769db00d583dbf656648218583a53dcac51254
>
> using the construction you described: lowercase SHA-256, one U+0020, the
> POSIX path, bytewise ordering by the UTF-8 path, U+000A after every line
> including the last, then SHA-256 over the resulting sequence. I used the
> fifteen vectors and excluded manifest.json.
>
> There is one input to that construction that is not fixed by the six
> points as written: the path root.
>
> To obtain 742509..., the paths have to be written as:
>
> src/vectors/NC-01.json
>
> so the effective root is cap-1/.
>
> If I use the same files and the same construction, but write the paths
> relative to cap-1/src/vectors/, the digest changes. That produces the
> ce88ddfc0feaec90... value you attributed to Emek.
>
> Nothing about the file contents or hashing procedure changes between
> those two runs. Only the namespace of the path changes.
>
> So I think the remaining repair is small: make the root part of the
> digest construction itself. Otherwise two readers can follow the stated
> procedure correctly and still hash different byte sequences because
> "relative path" has no fixed reference point.
>
> The bundle digest is where I stop being able to reproduce the object
> from the public bytes I can currently reach.
>
> At 0980d32, cap-1/src contains forty files. The bundle declares
> forty-two members. I looked in Certisyn-Inc/certisyn-drafts across all
> branches and the commit history, and in both archives carried in the
> tree, so if those members are public somewhere else, I have been looking
> in the wrong place and a pointer is enough to close that part of the
> question.
>
> If the difference is verify_bundle.py and mutate_bundle.py, the two
> filenames mentioned in your message, I could not locate either in the
> public tree I used for the reconstruction. If the bundle composition is
> different, then I am reconstructing the membership incorrectly, but that
> leads to the same practical problem: I do not yet have enough
> information to derive the exact forty-two-member set from the published
> material alone.
>
> That distinction matters to me more than the digest algorithm. The class
> digest is reproducible once its root is fixed. For the bundle digest, I
> first need to be able to reconstruct the member set itself.
>
> So my answer is:
>
> The class digest is reproducible. The construction itself clears the bar
> once the path root is made explicit.
>
> The bundle construction may clear it too, but I cannot verify that yet
> because I cannot independently reconstruct the exact member set that was
> hashed.
>
> If you publish the member list, or point me to the public object that
> defines those forty-two members, I can finish that check without making
> assumptions about the composition.
>
> Once that is pinned, I will run the class independently. I will freeze
> and hash my implementation and interpretation notes before the run, then
> report the source, environment, input digests and per-vector results.
>
> Best,
>
> Tiago Pinto
> Independent author and researcher, verifiable trust and AI governance
> https://donttrustverify.pt
>
>
> Em sexta-feira, 21 de agosto de 2026 às 15:07, Joel Hillier <jhillier=[e-mail]> escreveu:
>
> > Hi Walter, Emek, Tiago, Iman,
> >
> > Well, you all took me seriously when I said break it, and inside a day. Thank you. This is the loop working exactly as it should, and it's a far better use of a draft than agreement would have been.
> >
> > Emek has now run both his text-derived verifiers against the published fifteen, and that result is the news, so it goes first. Everything else is conceded below it, and the pinned bundle is at the end.
> >
> > Two readings of my document, run against my own class, agree on nothing.
> >
> > verdicts strict attribution
> > reader A 14/15 9/10
> > reader B 10/15 0/10
> > A vs B disagree on 15 of 15
> >
> > Fifteen for fifteen. Not a divergence on an edge case, a total divergence, and it has one root: the draft doesn't name the members.
> >
> > And the root under the root, which I would not have found by reading either. Emek's finding is that the published schema makes Section 9's MUST unsatisfiable. I checked it against the schema rather than take it on trust, and it reproduces exactly:
> >
> > subject.digest object, required [algorithm, value] states it
> > catalogue_digest string ^[0-9a-f]{32,128}$ cannot
> > withheld_digest string ^[0-9a-f]{32,128}$ cannot
> >
> > Section 9 says a producer "SHOULD use an algorithm that is unbroken at time of production and MUST state which was used." The pattern spans 32 to 128 hex, which is MD5 through SHA-512, so the algorithm isn't recoverable from the value. And both enclosing objects, `basis` and the `unexamined` item, are `additionalProperties: false`, so a producer who wants to comply has nowhere to put the statement. At two of the three sites where R3 and R4 rely on it, the schema forbids compliance with the draft's own MUST.
> >
> > That isn't a style inconsistency between two shapes. It's a normative requirement that cannot be met, and it's the root of the string-versus-object split Emek measured yesterday as eight disagreements in twenty. The first shape is the one to keep. `{algorithm, value}` goes to the other two, because the string form is exactly what the weakest-rung argument is about: a digest whose algorithm is unstated is a claim whose strength the reader can't evaluate.
> >
> > One correction to Emek's own finding, and it makes it worse rather than better. He writes that `subject` is an object with `kind` and `ref`, and that neither word occurs in the Internet-Draft. `ref` occurs zero times, which is right. `kind` occurs three times, and every one of them is `basis.kind` or the ordinary English word. So the schema requires `subject.kind`, and a reader who does meet the word meets it bound to a different object meaning something else. That's worse than absence, because absence at least reads as absence.
> >
> > Reader B took R0's "MUST name a subject" at face value, expected a string, and refused all fifteen under shape, which is why its attribution is 0/10. Emek's line for it is the one I'd keep: a class where every document fails the shape check tells its author nothing about the rules underneath. And `withheld_digest` cost reader A its only lost verdict on PV-04, because it looked for `digest` and for `hash`, the two names the prose suggests, and found neither. One member name, and a conforming document reads as non-conforming.
> >
> > NC-05, in my own negative control, without either of you arranging it. Reader A refuses NC-05 under R1 and R5 together. That's the analytic point arriving empirically inside the vector I wrote: a document violating R5's second sentence has already violated R1, so the mutant survives R5's removal. Emek reached it by reading and Walter reproduced it from the other side. My own control shows it, which is better evidence than either reading of it was.
> >
> > Walter found what sits underneath: whether 7.2's "the class MUST fail" incorporates 7.1's refused-by-the-rule-it-targets requirement. Attribution-aware, his R5 mutant dies. Mere-refusal, it survives, which is the survival Emek measured. Same protocol, two readings, opposite kill counts, and it's the load-bearing sentence of the whole conformance section.
> >
> > Emek, on the part of your message that wasn't about my draft. You could have pointed only outward and you didn't. An implementer reached 13/13 against your published vectors only after opening your source, and four things were nowhere in your documents: what the Ed25519 signature actually covers, that the sequence counter advances by exactly one, which absent fields are schema errors rather than tampering, and the order the checks run in. The last one is the one worth the thread: run the signature check first and a deleted receipt reports as a bad signature. True verdict, wrong reason. Check order is a wire fact and almost nobody writes it down. That's the same failure as `kind` and `ref`, and you found yours the same week you found mine.
> >
> > So the claims become, and all of it goes into `-01` with mechanism rather than as quieter sentences:
> >
> > Nine rules of which eight are exercised, R0 tested by nothing. Walter has that one concretely: silencing R1 through R8 kills the class eight times, silencing R0 leaves it passing, because no vector exercises shape alone, so a verifier with no shape checking whatsoever agrees on all fifteen. One R0-only negative control closes it, and until it exists the schema is normative and untested in the same breath.
> >
> > Agreement among implementations on the rules and not on the normative shape. That sentence is now measured rather than hedged.
> >
> > And the production-fixture claim comes out. Walter: Section 7.4's fixtures contain no `cap/1` document at all, both are `certisyn.verdict-document/1.0.0` hosts carrying the older coverage shape, and grep for `cap/1` returns nothing in either. So "correctly refused under R1" has no referent in the artefacts it cites. Your line for it is right: the claims most likely to be false are the ones nothing was pointed at, and these were never pointed at a CAP-1 verifier.
> >
> > The denominator has two doors, not one.
> >
> > Walter moves the edge after the answers are known: 500 checks, forty ugly, declare 460 on the `declared` basis, eight for eight, `complete: true`, perfectly accounted, about a population drawn after the fact. His sentence goes in the draft: the information didn't survive being moved from the remainder to the denominator, it just moved.
> >
> > Emek then leaves the edge alone and moves the claim instead. A stratum states which classes of claim it supports and nothing requires the cited claim to fall inside what it states. Five TLS configuration checks declaring `supports: ["absence-of-any-vulnerability"]` and bounding "no vulnerability was observed in the release" is conforming under both his verifiers. R8 checks presence. The failure it names is containment.
> >
> > The R7 incentive is the third face. A check you expect to error is safer never dispatched, because running it and erroring costs `complete` while declining to run it keeps `complete` with a tidy disposition. It rewards not looking, and nobody has to write a false statement to use it.
> >
> > The closed vocabulary isn't closed, and you found the boundary from four directions. `not_sampled` and `not_yet_due` from Walter, and his R4 argument settles the first: if an enumeration must state its method because the method determines what the enumeration could not see, then sampling is such a method and the undrawn units are none of the eight. `access_denied`, because landing an active refusal beside a missing licence loses the most interesting event in the run. `aborted_before_dispatch` from Tiago, with the sharper half being that `integrity.complete` speaks only of dispatched units, so a clean operator cancellation keeps `complete: true` while part of the eligible population was never reached. And Emek's unit that was dispatched, ran, terminated normally and determined nothing, which `failed` misdescribes and `unavailable` misdescribes and `examined` counts anyway.
> >
> > Emek, you declared an interest before anyone could notice it. That case is right on its merits and I'd have reached it more slowly without you naming it. `superseded` I want and am not adding yet, because the successor pointer is a different shape from the other dispositions and I'd rather get it right than get it quickly.
> >
> > The basis trichotomy, and two repairs that aren't alternatives. Walter and Tiago reached the derived population independently, and Tiago's note is the one I'd keep: a composable or `derived` basis preserves how the population was obtained, Walter's enumeration digest fixes which population was obtained, and neither substitutes for the other because a digest of a set doesn't let anyone re-derive it without the source and the predicate. Both go in. Walter's re-derivability axis then replaces the trichotomy: external artefact, internal artefact, or none. That partition doesn't leak, and `declared` comes to mean exactly one thing. The argument that convinced me is that the careful ones currently get punished, and a vocabulary that penalises rigour gets routed around by the people you most want using it.
> >
> > `withheld` is incoherent, and the vectors already disagree with the text. Tiago has it analytically: Terminology says a disposition is why a unit was not examined, Section 6 says `withheld` is examined, Section 4.1 puts it in `unexamined`, and R1 counts. No count assignment preserves all four. Plus R2 requires every unexamined entry to name a unit while the privacy rationale exists so a sensitive unit needn't be named. Walter settled it empirically in both halves: PV-04 carries eligible 7, examined 4, unexamined 3 with the withheld unit among the three, so the class counts `withheld` as unexamined against Section 6, and the withheld entry names its unit, `selector.ts`, against the privacy rationale. The vectors have been practising the repair on one axis and refusing it on the other. Tiago's two-axis split, examined and disclosed, is the fix.
> >
> > The pinned bundle. Built, not promised.
> >
> > class digest 742509bba744a1b6bf57a057a7769db00d583dbf656648218583a53dcac51254
> > bundle digest 7e3e54963de75316e41a957447b290739bfac5c408f9d973a66f04cf7313dca7
> > MANIFEST.json 0637ea1db3d330aa616e20c1727a191f1065b250e8f2f2df59db2345fe32ce59
> > archive 680e7d20862fee173c568cdf2f384e5bfa0d4fb07df614fc48b22f438fd241ad
> > members 42, of which 15 are the conformance class
> > source Certisyn-Inc/certisyn-drafts @ 0980d3201aa2caab3cbad5c6e9bc99b422370b43
> >
> > Walter, both digests you pinned reproduce exactly against that tree, `170aa81e...` for the vector manifest and `4453f216...` for the schema.
> >
> > Emek, one question about your reproduction, and I think it's a real hazard rather than a nitpick. You wrote that Walter's digests reproduce "exactly once line endings are normalised." The bundle pins SHA-256 over stored bytes with no normalisation, deliberately, so if your checkout is translating line endings then the bundle digest will not reproduce for you as specified, and the artefact built to end this class of ambiguity has one in it. Can you tell me whether that was a checkout setting on your side or a genuine difference in the stored bytes? If it's the second, the bundle is wrong and I'd rather know now.
> >
> > Your class digest doesn't reproduce, and that turned out to be the most useful thing in the exercise. "SHA-256 over the sorted per-file sha256 list of the fifteen" fixes the information and not the representation. I tried forty-four readings of that sentence against your value and none reproduced it: sorted by path or by digest, joined by newline or nothing or comma or space, with and without a trailing byte, hex or raw octets, with and without the filename, in `sha256sum` order or reversed, over fifteen files or sixteen, over stored bytes or canonicalised JSON.
> >
> > That isn't a criticism of the proposal. It's the proposal's own argument arriving one level up, and it's why the bundle specifies the construction to the octet instead of describing it. Six points, each naming a byte: enumerated member set never inferred from a directory listing; SHA-256 over stored bytes with no normalisation; ordered bytewise over the UTF-8 path; one line per member as lowercase hex, one U+0020, POSIX relative path; joined with U+000A including after the last; SHA-256 over that sequence.
> >
> > One departure from your construction, with the mutant that earns it. The digest is over path-and-digest lines rather than over digests alone, because a list of content digests commits to the multiset of contents and not to which file holds which. The bundle's mutation test swaps the contents of NC-01 and NC-02 and prints the path-blind digest either side:
> >
> > path-blind class digest, before: 47314682fddd39da
> > path-blind class digest, after : 47314682fddd39da
> > UNCHANGED, so a digest over content alone cannot see this
> >
> > The bundle carries its own verifier and its own mutants. `verify_bundle.py`, 42 members, 0 failures, and it reports every failure rather than stopping at the first, which is Emek's short-circuiting finding applied to the tool. That finding deserves its own line, because it generalises past both our drafts: refused by the rule it targets is only checkable against a verifier that reports every refusal. Against a short-circuiting one it's a fact about check order.
> >
> > `mutate_bundle.py`, four mutants, four killed, and the crediting rule there is a set rather than a singleton, for your reason rather than mine. Altering a vector must move its member digest, the class digest and the bundle digest together, so a mutant cannot produce exactly one failure. That is NC-05 again, in a different tool, within an hour of reading your report on it: a protocol demanding exactly one broken rule per control cannot be met where the rules entail one another. I hit it building the thing that exists because you found it.
> >
> > Standard library, no network, no arguments. Tiago, that should meet your bar; if it doesn't, say where and I'll fix the bundle rather than argue.
> >
> > Tiago's freeze discipline, agreed and extended: you, Walter and Emek's two readers each hash implementation and interpretation notes before seeing another's, then run against this. Agreement is then worth something and disagreement is worth more.
> >
> > Iman, thank you for the priority note, and it wasn't necessary from my side. An additional implementation is exactly as useful as a fourth one, and PR #630 written from `-00` is another reading of the same text by someone I haven't spoken to. Yes to the schema-shape and examined-set negative controls against the pinned commit, and they can now cite a class digest rather than a commit.
> >
> > One thing your break has already changed elsewhere. I read Walter's first message, went looking for the same shape one level up in the other draft, and it was there. ARP's evaluation sweeps carry an accounting identity over a bracket of ledger sequence numbers and check that nothing was skipped inside the bracket. Nothing required the bracket to abut the previous one, so a sweep could be complete over an interval beginning after the entries an operator would rather not have examined, and every rule passed. Now fixed: brackets must abut, and any range deliberately not examined is individually disposed with a registered reason, where the registry refuses a reason the reader can't check for itself. Walter's sentence is quoted there with his name on it.
> >
> > Different subject matter, same defect, found by reading a review of a different document. Which is the strongest argument any of us has made for the shared substrate, and it wasn't made in the abstract.
> >
> > Keep going. This is the most useful day either draft will ever get.
> >
> > Joel
>
> --
> SCITT mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]