Richard Sutton, author of the 2019 essay The Bitter Lesson, stated that raw compute is insufficient for achieving intelligence. He argues that AI science requires new methods, as current large language models fail to learn continuously after training and rely on limited synthetic data.
Sutton advocates for the development of agents with world models that learn from experience. Jerry Tworek, founder of Core Automation, supports the view that while scaling is significant, the choice of algorithm becomes increasingly critical as compute resources grow.
The debate has influenced the emergence of new AI startups. Frontier Radar #5 identified 45 new labs, including Sutton’s Oak Lab and Tworek’s Core Automation, that are exploring alternatives to current pretraining and reinforcement learning techniques.
Conversely, industry leaders like OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei maintain that LLM scaling remains effective. Altman cites an OpenAI model that recently solved a long-standing mathematical conjecture as evidence of continued progress.
