Capability plateau

Hits directly

• RL on checkable taskscapability · Built today
• Robust generalisationcapability · Emerging

Signs

In late 2024 labs reported diminishing returns from bigger pretraining runs, and reasoning models trained with reinforcement learning restored progress. Sutskever says the field is moving from an age of scaling to an age of research, though he later added that scaling will keep improving models.

What would soften it

New training paradigms, such as continual learning, arriving before the current one runs out.

Cut off 55

Slowed 25

Sources

1. 01Platformer: AI companies hit a scaling wall
2. 02Sutskever: a new learning paradigm is necessary