Reliability on repeat
Getting the same task right every time, not once.
capabilityEmerging Open on the map →
Facts
- Lane
- Models and thresholds
- Wave
- 6 of 12 (build order, not a date)
- Needs
- 0 boxes all the way back through hard links
- Unlocks
- 14 boxes downstream through hard links
- Leverage
- 2.3 points · stands in the way of 12 of 53 stages · rank 2 of 57
- Cluster
- Agents at work
Evidence as of October 2026
On τ-bench, GPT-4o succeeded on over 60% of retail tasks but on under 25% when it had to succeed eight times in a row. Success rates also decay with task length.
Around it
Loops and off-ramps
Openings researchers' judgement, October 2026 scouting
Startup room 100/100 · rank 1 of 124 Research room 100/100 · rank 1 of 124
- Depth
- deep
- Players
- Braintrust ($80m B, $800m); LangChain ($1.25bn); Patronus ($50m B); LMArena ($1.7bn); Princeton CITP research
- Wedge
- Runtime verifier and recovery layer that raises effective pass^k on a fixed model for one workflow; research on methods that raise consistency and replay-proof benchmarks. startup