Agent Action Capsule · build your own evidence layer
b9 / 11 · the loop: judge

Judging: a separate step from evidence

Seam ③ Check/judge. Build: judgment records kept apart from evidence records. Conform to: draft-mih-agent-evidence-layer-00 §4.1, §5.

Evidence says what was done, by whom, and that the record has not changed since. A judgment says whether it was right. Keep the two as separate records: a verifier can recompute what the evidence supports from its bytes, while the judgment stays labelled as one, naming who made it and on what basis.

An act record, a verifier that recomputes its assurance, three ways of judging, and the judgment record they produce Top left: an act record. Step 1: it is sealed with what was done, its own half of the exchange and its own assurance block. Step 2: the assurance block claims full_bilateral, but only one half is present. Step 3: a verifier recomputes from the bytes: derived unilateral_fallback, finding assurance_overclaim; the claim is left in the record. Step 4: judging is a new record citing the act by digest; the act is not edited. Middle row, three ways of judging. Step 5: rules, tier recomputed, anyone with the inputs re-runs it; designed. Step 6: a judge model, pinned by model id, prompt digest and sampling; each case met, not_met or not_evaluable, with its rubric cited; blind human ratings and calibration agreement as k of n; designed. Step 7: twins and a referee; a verdict record with epistemic type adjudication citing both halves, signed by the referee's key; built, not live. Step 8: each route ends in a judgment record whose epistemic type says how it was known (derived_metric, semantic_judgment or adjudication), fixed at commit and never upgraded, citing what it judges, with its judge and basis. Step 9: a verifier checks the signature and what it cites, not whether the judgment is right; was the answer right stays a judgment, never folded into evidence. EVIDENCE WHO JUDGES JUDGMENT Act record what the agent did action_id · timestamp cross_party: own half assurance: { … } claims: full_bilateral only its own half present Verifier recomputes from bytes claimed: full_bilateral derived: unilateral_fallback finding: assurance_overclaim the claim stays in the record Judging: a new record cites the act by digest the act is never edited act ← judgment 1 · Rules 2 · A judge model 3 · Twins + referee (mesh) tier: recomputed DESIGNED a fixed function of the cited inputs; no model, no human in the loop anyone with the inputs re-runs it tier: judged DESIGNED evaluation-report/v1 pinned: model, prompt, sampling met | not_met | not_evaluable blind human ratings · k of n tier: judged BUILT, NOT LIVE referee re-answers one pair type: adjudication verdict: contradicted:B signed by the referee's key Judgment record epistemic_type: derived_metric | semantic_judgment | adjudication set at commit, never upgraded; cites its target by digest names its judge and its basis A verifier checks the signature, what it cites, its type not that the judgment is right “Was the answer right?” = judgment, never folded into evidence

1What it is

Judging is the step after recording: something decides whether an act was right, and that decision becomes a record of its own, linked to the act by digest instead of written into it.

The judge depends on the work: rules a program can re-run, a judge model pinned to its model, prompt and sampling settings, with blind human ratings beside it, or, on a mesh, twins and a referee that signs a ruling on one pair.

A verifier recomputes what the evidence itself supports, reporting any assurance a record claims above that, and treats a judgment as what it is: a signed statement by a named judge, which it can check for integrity but not for correctness.

2Design choices

Chosen: a judgment is a new record carrying an adjudicates link to the act's digest; the act is never touched.
Rejected: writing a verdict field into the act record after the fact. “A link is itself part of a committed record and MUST NOT mutate its target.” (draft-mih-agent-evidence-layer-00 §5 Typed Links)
Chosen: the kind of statement a record is (epistemic_type) is set once, at commit, and never upgraded; a record with an adjudicates link MUST be of type adjudication.
Rejected: letting a later signature or checkpoint turn a judgment into something that reads like an observation. “epistemic_type is assigned once, at commit, and is never upgraded.” (draft-mih-agent-evidence-layer-00 §4.1, §5)
Chosen: the verifier recomputes each assurance value from the bytes and reports any claim above it. An overclaimed effect_mode fails verification; an overclaimed attestation_mode, ledger_mode or cross_party_rung is reported as a finding next to the derived value.
Rejected: trusting the record's own assurance block. “A producer MUST NOT record an assurance mode it did not achieve; a verifier rederives each mode from the evidence present and reports any overclaim.” (draft-mih-scitt-agent-action-capsule-05 §5.4, §5.4.1, §6; verify.py check 7)
Chosen: two answers kept apart on every claim: whether enough evidence existed to judge (SATISFIED / GAP / INSUFFICIENT / UNKNOWN), and only then the judgment (met / not_met / not_evaluable).
Rejected: one pass/fail figure, and using not_evaluable for missing evidence. “Every claim carries two answers that are never merged.” (agent-action-capsule spec/evidence-result-v0.md §1, design specification)
Chosen: say how a result was produced: recomputed (a fixed function of cited inputs, no model or human) or judged (a model or a human applying judgment, pinned).
Rejected: presenting a model's reading with the same standing as a recomputation anyone can repeat. (spec/evidence-plan-ir-v0.md §4.2)
Chosen: a judge model is pinned in the record: model_id, prompt_digest, optionally axes_digest, and sampling settings as integers, strings or booleans; each case cites the acts it rests on and the rubric (method), which is required.
Rejected: an unpinned judge, sampling settings as floats that can drift in serialization, and a confidence figure: that field has no counterpart in the family. (spec/judge-record-family-v1.md §2, §6, §9)
Chosen: humans rate cases blind (blind is fixed true), and a judge's calibration agreement is stored per clause as k of n.
Rejected: a blind: false rating in the same shape, and a stored rate: a reader divides k by n when reading. (spec/judge-record-family-v1.md §5, §8; vectors neg-blind-false, neg-clause-with-rate-field)

3What it shows / what it does not

Shows

  • Which assurance a record's own bytes support, and any claim above it, with the claimed and derived values side by side.
  • That a judgment exists, who signed it, what it cites by digest, and that it has not changed since it was signed.
  • What kind of statement it is (semantic_judgment, adjudication), fixed when it was committed.
  • For a pinned judge model: which model, prompt and sampling settings produced it, and which rubric each case used.
  • For a calibration summary: the judge's agreement per clause, as k of n.
  • For a referee's verdict: which twin its re-answer did not agree with, under the referee's signature.

Does not show

  • That the judgment is correct. A verifier can recompute a recomputed result; a judged one stays the judge's statement.
  • That the judge is independent of the party judged; that is a claim about who runs it.
  • That an act was right because its record verifies. The record shows what was done, not whether it was the right thing.
  • Anything about cases the judge did not see, or that a sample was not chosen to flatter.
  • A single figure for the agent. Judgments stay per case, and agreement stays k of n.

4Example

The verifier reports an overclaim

A conformance vector: the record claims full_bilateral, but its cross_party block has only the initiator's half, with no counterparty_ref. Trimmed input, then the verifier's expected answer:

"assurance": { "attestation_mode": "self_attested", "cross_party_rung": "full_bilateral",
               "effect_mode": "not_applicable", "ledger_mode": "standalone" },
"cross_party": { "initiator_ref": "1111111111111111111111111111111111111111111111111111111111111111" }

"derived":  { "cross_party_rung": "unilateral_fallback", ... },
"findings": [{ "check": 7, "code": "assurance_overclaim", "severity": "info",
  "detail": "claimed cross_party_rung 'full_bilateral' but verifier derived 'unilateral_fallback' (§5.3 Cross-party assurance)" }],
"ok": true

Source: action-state-group/agent-action-capsule:vectors/capsule/neg-cross-party-overclaim/{input,expected}.json @ 2ef8aeb49e2334e24c404bd62d3fdb161770873c (also in tag python/v0.10.0). The detail string's §5.3 is a stale section number; the text is §5.4.1 in both -04 and -05.

A judgment record: the referee's signed verdict

The verdict a referee seals on a mesh twin pair, from the plugin's test fixture (placeholder node ids), trimmed:

"provenance": "referee",
"model_attestation": { "compute_attestation": {
  "epistemic_type": "adjudication",
  "adjudication": {
    "schema": "capsule-emit-mesh/adjudication/v1",
    "source": "twin_comparison", "capture_method": "deterministic_replay",
    "verdict": "contradicted:bbbb…bbbb", "divergence_index": 3,
    "half_a_capsule_id": "7dc9427b6df5ff222617fb721f25d3c54aebe32682329bb7075902336437fb06",
    "half_b_capsule_id": "b80db10999779aa17c8fcf7006a57aeaecbc81f5e9de3ecea33e9365c2bfd4d9",
    "referee_answer_digest": "0242bc9526adbb014565b8175ff134e2ba8731f4a13cab2d1b4d8aff8727586e",
    "twin_bracket_id": "bracket-1" } } },
"assurance": { "attestation_mode": "self_attested", "effect_mode": "not_applicable", "ledger_mode": "chained" },
"disposition": { "decision": "accept", "approver": "policy", "human_disposed": false, "verdict_class": "assessed" },
"chain": { "parent_capsule_id": "7dc9427b…fb06", "relation": "adjudicates" },
"key_id": "6a9a93a3af20ca17d85764e3206fd1d0417ea943f53f07157792299ba9b6f033"
FieldTypeMeaning
epistemic_typestringadjudication: a ruling, not an observation. Set when sealed; nothing later changes it.
source / capture_methodstringHow the ruling was reached: a twin comparison, re-answered by deterministic replay.
verdictstringcontradicted: plus the node the referee did not agree with, or corroborated, inconclusive, not_comparable.
divergence_indexintThe position of the earliest word where the twins' answers split.
half_a / half_b_capsule_idhexThe two twins' own signed records the ruling covers, by id.
referee_answer_digesthexDigest of the referee's own re-answer, the basis of the ruling.
effect_modestringnot_applicable: a verdict changes nothing in the world.
verdict_classstringassessed: the plugin's own value for a judging record. It is not in the registry, so a verifier reports it as an unknown value, informationally.
chain.relationstringadjudicates, chaining to half A. A legacy spelling, not registered: the registry reads it as confirms and states it is unrelated to the evidence layer's adjudicates link.
key_idhexThe referee's signing key. A holder checks the signature against it; the signature shows who ruled, not that the ruling is right.

Outside the mesh, the same ruling has a published shape, adjudication/v1, and a judge model's per-case result has evaluation-report/v1; spec/judge-record-family-v1.md §9 maps the mesh block's verdict, halves, referee, margin and divergence fields onto it; pipeline bookkeeping such as schema, source and capture_method has no counterpart. The plugin still seals its own block.

Source: Mesh-LLM/capsules:crates/evidence-plugin/tests/fixtures/referee-verdict.json @ b4e24453f572b2d9fe8f2cefbd0f35a5eb54b0ad (tag v0.1.4)

5Status

ItemStatusEvidence
The verifier recomputes assurance and reports overclaims (check 7: effect_mode fails verification; attestation_mode, ledger_mode, cross_party_rung are reported as findings)built, releasedagent-action-capsule python/agent_action_capsule/verify.py in tag python/v0.10.0 (f3dcc32213396f71218e0e33539869f99cff9c9e); Go go/verify/verify.go and TypeScript ts/src/verify.ts on main 2ef8aeb49e2334e24c404bd62d3fdb161770873c; vector neg-cross-party-overclaim.
A claimed time rung above what the record's references support fails verification (check 9, provenance_time_rung_overclaim)built, releasedSame verify.py in python/v0.10.0; provenance-mode-vectors/neg-provenance-mode-time-rung-overclaim @ 2ef8aeb4.
epistemic_type fixed at commit; an adjudicates link only on an adjudication recorddesignedSpecified in draft-mih-agent-evidence-layer-00 §4.1, §5. A conforming book refuses the mismatch.
Referee-signed verdict on a mesh twin pairbuilt, not livecapsules v0.1.4 (b4e24453): crates/evidence-plugin/src/referee/, crates/evidence-plugin/src/adjudication_records.rs. CHANGELOG.md 0.1.4: “A referee verdict on live hosts waits on 0.1.5; until then it is covered by tests only.” See b4.
Judge record family: evaluation-report/v1, adjudication/v1, human-rating/v1, calibration-summary/v1 and four more shapesdesignedagent-action-capsule spec/judge-record-family-v1.md (“Design specification, pre-Internet-Draft”), schemas/judge/*.json, vectors/judge/ (one positive, one negative per shape) @ 2ef8aeb4. Schemas and vectors are published; no open-source producer emits these shapes yet.
Evidence Result: sufficiency and verdict kept apart, tier recomputed / judgeddesignedspec/evidence-result-v0.md, schemas/evidence-result-v0.json, spec/evidence-plan-ir-v0.md §4.2 @ 2ef8aeb4; design specifications, pre-Internet-Draft.
A rules judge emitting tier: recomputed resultsdesignedspec/evidence-plan-ir-v0.md §4.2 @ 2ef8aeb4 defines the tier; no open-source implementation.
The mesh plugin sealing adjudication/v1 instead of its own blocknot builtThe v0.1.4 fixture above carries "schema": "capsule-emit-mesh/adjudication/v1" and the legacy chain.relation spelling adjudicates (spec/REGISTRY.md, deployed legacy aliases); judge-record-family-v1.md §9 is the field map for the port.

6Specs

7What you need before the next step