Map/Measurement and understanding/Wave 2

Calibrated abstention

Models that abstain or flag uncertainty instead of guessing, scored by evaluations that penalise confident errors.

Facts

Lane: Measurement and understanding
Wave: 2 of 12 (build order, not a date)
Needs: 2 boxes all the way back through hard links
Unlocks: 0 boxes downstream through hard links
Leverage: None
Cluster: Oversight and alignment
Found by: side effects lens

Evidence as of October 2026

On SimpleQA, OpenAI's o4-mini abstained 1% of the time and was wrong 75%, against 52% abstention and 26% errors for gpt-5-thinking-mini at similar accuracy (OpenAI, 5 Sep 2025, vendor comparison).

01Why language models hallucinate (OpenAI, Sep 2025)

Around it

Needs · 2

Real-world task evalscapability · Emerging
Reliable verifierscapability · Built today

Unlocks · 3

Reliability on repeatcapability · Emerging · helps, not required
Scalable oversightcapability · Emerging · helps, not required
Liability and insurancecapability · Emerging · helps, not required

Loops and off-ramps

Trust and delegationreinforcing loop

Openings researchers' judgement, October 2026 scouting

Startup room 39/100 · rank 73 of 124 Research room 50/100 · rank 62 of 124

Crowding2.5 Lab overlap4 Small-team fit3.5 Buyer today3 Research openness4

Depth: light
Players: OpenAI (abstention-aware evals); Oxford BAS metric; Goodfire ($150M B at $1.25bn, interpretability); hallucination-detection vendors (AIMon)
Wedge: Abstention and selective-prediction evals with asymmetric scoring for multi-step agent tasks and a firm's own question sets, where confident errors are the real cost. research