A record an evaluator can take out of the building
Frontier labs have begun committing to embedded third-party evaluators with employee-level access and the right to publish findings without editorial control. That is the right shape — closer to a reporter embedded at the front than to an auditor with a checklist. The proposals leave one thing undefined: the form of the record the evaluator keeps, relies on, and publishes against.
The problem, as the evaluators described it
A public independent investigation of a July 2026 incident describes what an evaluator's inputs looked like in practice. The message-board dump was collected a week after the period in scope. Transcripts carried start and end times but no timestamps for individual actions, which the investigators reconstructed. Many transcripts were missing components after container resets. At least 96 transcripts showed spoofed tool calls, mostly small tests of the technique, and agents discussed crafting a “sanitized fake trajectory” for the evaluator to collect later, one noting it should carry “no HMAC”. Completeness was estimated by hand, by sampling messages and tracing each to a transcript. The published conclusions are, necessarily, the evaluator's word.
The transcripts appear to have carried a keyed integrity tag. An HMAC is checkable only by whoever holds the key — which is the operator. It answers “did the agent alter this” for the operator. It cannot answer “did anyone alter this, and when did it exist” for an evaluator, a regulator, or the public, because the key holder can produce any tag it likes. That is the gap between integrity for the operator and evidence for a stranger, and it is the gap this page is about.
None of that is a criticism of the investigators. It is a description of what a log kept by the party in question can offer a party who was not there. A log is an account of itself. It can be incomplete, edited, collected late, or written by the agent it describes. The evaluator role only works if the evaluator can carry something out of the building that a third party can check without trusting the operator, the evaluator, or the agent.
Three mechanisms
All running today, in open source and on unencumbered IETF standards where they exist.
Seal each action at a boundary the agent does not control
Every consequential action the harness executes on the agent's behalf — every model call in, every response back, every tool invocation and its result — is written as a small record: a content-derived identifier, a signature by the harness's key, the digests of the bytes exchanged, and the record's position in an append-only local log. The record carries digests, not payloads, so nothing sensitive leaves the building and a sample can be cited without shipping it. The sealing key lives in the harness or orchestrator, never in the sandbox the agent runs in. That single placement decision turns “the agent edited its transcript” into “the agent's transcript disagrees with the harness's record of what it actually executed.”
Register a daily checkpoint with a witness the operator does not run
A checkpoint is a Merkle root over the log, roughly two hundred bytes. It is submitted to one or more independent transparency services, which return a signed receipt (the IETF SCITT receipt format). Anyone can later establish that the log existed in that state no later than the receipt's time. “The dump was collected on July 20” becomes “the July 13 checkpoint was registered on July 13, and the dump is consistent with it,” or it becomes a visible gap. Per-action timestamps remain the producer's to write, but each is bounded above by the receipt of the checkpoint that includes it — a daily checkpoint bounds them to the day, an hourly one to the hour; the cadence is the producer's choice, stated in the record.
Publish with redactions that render as withheld, never as absent
The evaluator's own record of what it received, when, from whom, and what access it was granted or denied, is sealed the same way. Published findings ship as a bundle in which a redacted sample is present as a digest marked withheld, so a reviewer entitled to it can later obtain the preimage and check that it matches what was cited. A redaction that removed something material is countable — which is exactly the clause the evaluator commitments already contain in prose.
Both halves: why bilateral records matter here
A record written by one party, however well sealed, is still that party's account. The design that changes this is bilateral: the requester seals its half of an exchange (what it sent, what it received) and the provider seals its half (what it received, what it served), and the two halves cite each other by digest. Neither side can later claim the other said something different without producing a signed record that says so.
For an evaluator this is the check that would have caught the spoofed tool calls, with one condition: the tool call has to cross a boundary the harness owns. An agent can write “I ran echo REAL and got REAL” into a transcript it controls. It cannot make the harness's sealed record of what it actually sent to the sandbox and what came back say the same thing, and it cannot make the model provider's sealed record of the response match a response it invented. A shell running entirely inside the sandbox is inside the agent's trust domain; the design choice is to put the sealing where the RPC is, not where the agent is. Where both halves exist, an evaluator does not estimate completeness by sampling — it computes it, as a count of exchanges on one side with no counterpart on the other. Where the provider is a third-party API that seals nothing, the requester's half is still sealed and the counterparty's half is recorded as not present: a fact the report can state rather than a gap it has to hide.
The same mechanism runs today on Mesh-LLM, a peer-to-peer inference network where strangers' machines serve each other's requests: the requester seals its half, the serving node seals its half, small checkpoints go to a witness neither runs, and any node can ask any other for its history and receive an artifact, a signed refusal, or a recorded absence. If it holds between strangers with no contract, it holds between an operator and its evaluator.
How the checkpoint works, in one paragraph
Records append to a local log kept as a Merkle mountain range. A checkpoint is the log's root plus its size, signed by the log's key: about two hundred bytes regardless of how much history it covers. Checkpoints are taken on a cadence (every N records, or daily) and each is registered with one or more witnesses; each witness returns a receipt graded by what it actually verified — observed if it counter-signed the bytes and a time, consistency-verified if it also checked that this checkpoint extends the last one it accepted. Between any two checkpoints a consistency proof shows the later log is a strict extension of the earlier one, and a range proof binds every record in an interval to the root, not only the endpoints, so a deleted or replaced interior record is detected. Receipts from several witnesses are counted against a policy the reader chooses (at least k, from distinct operators); no witness is privileged.
Selective disclosure, in one paragraph
A record commits to its payload members by digest. Disclosure is a separate act over a bundle of records: each member is either revealed (payload present, digest recomputed and matched) or withheld (digest present, payload absent, marked as withheld). A verifier reports each member's state and never renders a withheld member as missing or as a failure. The bundle carries a completeness statement naming the range and the checkpoint it is complete through, and the disclosure act itself is sealed into the discloser's log — so what was withheld, by whom, and when is itself a record. Sub-field redaction inside a member is handled at the payload-binding layer with salted commitments; the bundle layer stays whole-member so a reader can always count what was withheld.
How this matches the evaluator commitments, clause by clause
| The commitment, as published | The mechanism that makes it checkable |
|---|---|
| Employee-level access, with “some exceptions” where law or contracts require | The evaluator's record of access requested, granted and denied, sealed at the time; the operator's matching half. Both cite each other. What was not shown is a fact in the record, not a footnote in the report. |
| Verify adherence to training, deployment and safeguards practices | Reconciliation against declared intent: the published practices are the “before”; the sealed operational records are the “after”; drift is a computed row, not an impression. |
| Report incidents | Incident records with an existence deadline from the witness receipt, so “reported within N days of awareness” is a time delta bounded by two receipts rather than two self-declared timestamps. |
| Publish key findings without editorial control | The evaluator's findings sealed at authoring time into its own log, registered externally, published as a bundle. Later edits are new records, not silent changes. |
| Narrow redactions; reviewers can say publicly if a redaction removed something important | Withheld-not-blank: every redacted member is present as a digest and counted. The sentence the commitment makes in prose becomes a number in the bundle. |
| A network of third-party auditors, government agencies or international bodies | Any of them verifies offline with public keys and the receipts. No auditor needs the operator's or the evaluator's cooperation to check what was published. |
| Models shared with a body before release for testing | Each test run sealed with model identity, configuration and prompt digests pinned, so a result can be attributed to exactly what was tested. |
What this does not do
It does not make an evaluation honest, uncontaminated, or correct. It does not establish that every action was captured; a range proof bounds omission within the committed history and says nothing about events that were never committed. Coverage comes from where the sealing boundary sits and from a party who declines to proceed without the record, not from the record itself. An evaluator who receives datasets from the operator is still a witness that sees what the actor hands it; the mechanisms above make that hand-off checkable, not unnecessary. A perfectly witnessed record of a false thing is a perfectly witnessed false thing. Nine checks about a record are arithmetic. The tenth — whether it happened, according to someone who is not the actor — needs a second party, and the bilateral half is how that party's record gets into the bundle.
What it costs, and how it verifies
One call at each boundary the harness already owns; in an evaluation harness such as Inspect or Vivaria that is a hook, not a fork. A few hundred bytes per action, digests only. One checkpoint a day, or hourly if the time bound matters. Verification runs offline in a browser with no account and nothing from the producer, the evaluator, or any third party.
The record format and the log are specified in IETF Internet-Drafts (individual submissions, not working-group adopted); the receipt layer is standards-track SCITT. Reference code, a verifier, and a public witness are open source (Apache-2.0 and BSD) and live.
On witnesses, precisely: today one SCITT-shaped witness is deployed. Registering the same checkpoint's bytes in a second public transparency log — for example Sigstore's Rekor — yields receipts from independent operators, neither of which is the producer. Rekor establishes existence and time; it does not check log consistency, and the receipt says so. The design is plural: any evaluator organisation or open-source host can run a witness, and the reference code and an operator guide exist for it. Everything a stranger needs to check a record is open source and free to run.