Prerequisites II: Verifiers and Graders
Every loop on this site is only as honest as the component that renders its verdicts. This page defines that component, explains the training method that made verifiers the center of modern AI, and then covers the three ideas that decide whether a check can be trusted: it must be frozen, it is usually necessary rather than sufficient, and any check used as a target will eventually be gamed.
Verifiers and graders
A verifier renders a verdict on a piece of work: pass or fail, or a score. Verifiers come in a rough hierarchy of trustworthiness:
- Executable verifiers. A test suite runs. A proof-of-concept either triggers the vulnerability or it does not. A patch either survives re-attack or it does not. The verdict is a physical fact about the world, expensive to fake.
- Judged graders. A model compares the work against a reference or a rubric and assigns a score. Flexible, applicable where no test exists, and fallible in measurable ways. On two companion benchmarks built by Paper 3's loop for another organization (105 and 150 tasks, three runs per setting), up to one verdict in six flipped between identical runs of an LLM grader, and swapping the grader's model family moved scores by about 11.5 points while leaving rankings unchanged (paper-reported).
- Self-reports. The system grades its own account of its work. Paper 3's control experiments include a grader that scored the agent's description of its patch rather than the patch. On a 52-task control set it preserved the model ordering and was still rejected, because it saturated: everything scored between 0.90 and 1.00, so it discriminated nothing.
The hierarchy is not snobbery. An optimizer, human or automated, pushes against whatever renders the verdict. Executable checks push back. Self-reports fold immediately. Paper 3's broken-grader controls are consistent with that ordering: the self-report grader saturated, and a length-only score barely tracked capability. They do not measure the full hierarchy, because Paper 3's study cohorts had no executable tests to compare against. The ordering is a well-motivated working rule (doctrine), with Paper 3 supplying evidence about its lower rungs.
RLVR, in one section
Reinforcement Learning with Verifiable Rewards is the post-training method named by the Tulu 3 recipe (AI2, 2024; see the reading path, abstract-level). The pattern: train on prompts whose answers can be checked mechanically, a math answer that either matches or does not, an instruction constraint that either holds or does not, and use the binary verdict as the reward signal. No learned reward model, no human preference labels per example. The verifier's verdict is the training signal.
RLVR matters to this site for two reasons. First, it established the era's central pattern: if you can check it mechanically, you can optimize against it. The November papers import that pattern from weight training into harness space and benchmark space. Paper 1 optimizes harness code against verifier reward. Paper 3 validates benchmarks against ordering anchors. The logic is the same; the substrate differs. The two spaces page develops the contrast.
Second, RLVR is why the word "verifier" carries weight now. Optimization quality is capped by verdict quality. A noisy or gameable verifier does not produce a slightly worse model. It produces a model expertly optimized for the verifier's flaws. (That framing is the site's doctrine, and it is the premise every paper on this site builds its frozen check around.)
Why the check must be frozen
A frozen check is one the loop cannot edit, retune, or re-derive while it is being measured. Each paper freezes a different thing:
- Paper 1: graders, test suites, and logs live in read-only storage outside the agent's sandbox. The stated invariant: a self-improving harness must never control its own permission boundaries, its grading harness, or its audit logs.
- Paper 2: held-out task sets are scored before and after optimization and never feed any decision inside the loop. Only evolve-set feedback may influence proposals.
- Paper 3: the grader and the expected model ordering are frozen and explicitly excluded from the list of allowed repairs. The loop may change how tasks are sourced, how they are presented, even the pass threshold (last, and cautiously). It may not change what counts as correct, or which model is supposed to win.
The reason is not paranoia about malice. It is arithmetic about search. A loop that runs hundreds of candidate evaluations is a powerful optimizer, and an optimizer with write access to its own metric will find the metric's soft spots the way water finds a crack. Paper 1's failure taxonomy documents the observed versions, mostly drawn from work it cites: agents reading leaked reference solutions, weakening test assertions, suppressing error codes, deleting their own execution logs. Freezing is what converts "the loop improved the score" into evidence that the loop improved the thing.
Necessary versus sufficient
Paper 3's authors state the honest version of their own method: their anchors are necessary, not sufficient conditions for a valid benchmark. A benchmark that fails the anchors is certainly bad. A benchmark that passes has merely survived checks that bad benchmarks fail. It might still be unrealistic, mis-specified, or measuring the wrong skill. Their expert audit (30 tasks, one rater per task) found the point in practice: the main defect class in accepted benchmarks, tasks whose descriptions omitted details the reference solution contained, was invisible to the anchors, because under-specification lowers every model's score alike and leaves the ordering intact.
Generalize this, because it applies to every gate on this site. A gate is a filter for known failure modes, not a certificate of quality. The correct posture, and the one all three papers model, is layered: a frozen check to reject the provably bad, negative controls to prove the check still bites, and human review reserved for exactly the questions the check cannot answer. When someone claims a gate validates something, the precise question is: which failure classes does it reject, and which can it structurally not see?
Goodhart's law, applied to loops that grade themselves
Goodhart's law: when a measure becomes a target, it ceases to be a good measure. In this research family it is not an aphorism; it is a phenomenon with a lab instrument. SpecBench (2026, see the reading path, entry 17, abstract-level) splits each task into visible tests and held-out tests. Genuine solutions pass both. The gap between the two pass rates is the reward-hacking measurement. Across 30 systems tasks, every frontier agent tested saturated the visible suite while the held-out gap persisted, and the exploits included a 2,900-line program that memorized test inputs instead of implementing the system.
Now put a self-improving loop on top of a gameable measure and the dynamic compounds. A single agent might stumble into a shortcut. A loop searches for whatever raises the score, generation after generation, and archives the winners. Paper 1 calls this benchmark cheating and notes that outer-loop search amplifies whatever shortcut the sandbox leaves open. Paper 3 found the subtle version inside its own method: a broken grader that scored self-reports instead of code would pass its anchors if the pass threshold were set at exactly 94, and the threshold is a knob its loop is allowed to turn, last in the repair order, under human review. The authors published that blind spot themselves.
The defenses recur across the papers and are worth listing as a set, because together they are the practical content of "frozen check":
- Keep the final evaluation on tasks the loop never sees during optimization (held-out sets, hash-frozen holdouts).
- Put graders and logs where the loop cannot write.
- Run negative controls: broken graders, trivial tasks, shuffled labels. A check that never rejects anything is not a check.
- Record predictions before outcomes, so success cannot be retroactively redefined.
- Keep a human, or a statistical procedure with known error rates (TRUST), at the acceptance boundary.
- Attack the gate itself. Run an adversarial arm whose proposer is rewarded for passing the check without doing the work, and measure how often it gets through. No paper in this family did this (doctrine, drawn from a valid point in the adversarial review; see the synthesis). Fixed negative controls show a check rejects known-broken inputs. Only an optimizing adversary shows whether it resists the inputs nobody thought of.
The next prerequisite page turns to the instrument these checks usually protect: the benchmark itself.