Reliable verifiers
A trustworthy signal for whether an output is right.
capabilityBuilt today Built but scarce · 3 of 5 Open on the map →
Facts
- Lane
- Measurement and understanding
- Wave
- 1 of 12 (build order, not a date)
- Needs
- 0 boxes all the way back through hard links
- Unlocks
- 66 boxes downstream through hard links
- Leverage
- None: already built
- Cluster
- Science and medicine
Evidence as of October 2026
Jason Wei’s verifier’s law: how easy a task is to train AI on is proportional to how verifiable it is. Every task that is solvable and easy to verify will be solved.
Detail boxes 2
Training checks that models cannot game.
Environment builders consistently ranked robustness against reward hacking as the top quality criterion. Models have been caught checking future commits on SWE-bench Verified.
A training signal for subjective quality, such as voice and style, as dependable as unit tests are for code.
With objective errors removed, standard reward models (52.7%) and LLM judges (53.9%) predict human writing preferences at about chance, while reasoning reward models reach 81.8% (WritingPreferenceBench, Oct 2025, revised Jan 2026).
Around it
Nothing beyond the detail list. It can start at any time.