The Bitter Lesson

Published

Ironically, one of the most efficient strategies for building with AI is to wait for better models.

Researchers try to address shortcomings of AI models by constraining them from general purpose tools to specific purpose tools. The exact opposite approach eventually obviates their work as other researchers improve models by scaling compute rather than through better algorithms and clever constraints.

This is relevant to today’s approach to AI using large language models. The models improve incredibly fast and one side-effect of that is foundation model providers regularly wipe out companies trying to build specific applications once the model becomes good enough.

Read The Bitter Lesson by Rich Sutton and AI Founder’s Bitter Lesson from Lukas Petersson.

See also: