Contamination-resistant evals
Benchmarks in coding, maths, science and agent work that models cannot have seen or looked up.
capabilityEmerging Open on the map →
Facts
- Lane
- Measurement and understanding
- Wave
- 6 of 12 (build order, not a date)
- Needs
- 0 boxes all the way back through hard links
- Unlocks
- 0 boxes downstream through hard links
- Leverage
- None
- Cluster
- Agents at work
Evidence as of October 2026
SWE-bench Verified is saturated at around 95% and was deprecated by OpenAI in February 2026; on the harder SWE-bench Pro the same leading model drops to 80%. Epoch reports models searching online and checking future commits to game benchmarks.
Around it
Needs · 0
Nothing beyond the detail list. It can start at any time.
Unlocks · 1
Loops and off-ramps
Openings researchers' judgement, October 2026 scouting
Startup room 52/100 · rank 30 of 124 Research room 35/100 · rank 105 of 124
- Depth
- light
- Players
- LiveCodeBench; SWE-bench Pro; Datacurve DeepSWE; WorkBuddy-Bench; data vendors
- Wedge
- Renewable private task supply plus contamination detection, sold to labs. startup