AI Wants to Eat the World. The World Has Other Plans.
The world is too complex and unpredictable for AI development to continue uninhibited
AI development can be seen as a sort of flywheel.
Cumulative improvements takes place at different parts of the AI system development lifecycle: pre-training, post-training, inference and agents.
With pre-training, it has been observed that if you increase the size of the model, the amount of data the model is trained or the amount of compute used for training, the performance of the model also increases. It is based on an observed strong correlation between these different aspects of the training operation and model performance.
Post-training focuses on techniques to improve the capabilities of the base model produced after pre-training is complete. Think reinforcement learning, fine-tuning etc.
Scaling laws have also been found to exist at inference. With this, the more time a model spends processing the input (i.e., ‘thinking’ or ‘reasoning’ about an answer to a prompt), the better the response it produces.
And then agents are models forming part of wider systems built to perform certain kinds of tasks in a given domain. They interact with the wider world through the use of tools, data and feedback loops to complete tasks with little human supervision.
All of these things link together to form a flywheel for AI:
Improved pre-training leads to improved base models
Improved post-training leads to improved foundation models
Improved reasoning leads to improved agentic capabilities
Deployed agents generate new data that is used to train future models
If this flywheel is allowed to persist, then maybe we get great leaps in AI capabilities.
But the reality is that this flywheel will not always be spinning. Or at least it will not be spinning as quickly.
I think a lot of people in the ‘AI will disrupt everything’ crowd forget about the real frictions and constraints that cannot always be engineered around and will make AI’s diffusion across the economy much slower and less disruptive than they predict.
The world is far less predictable and far more complicated than people realise. Anyone is capable of overfitting their perception of the world to their own experiences, but what they experience is merely a fraction of what happens in the world.
So the AI flywheel is limited by real constraints that should not be forgotten, and you can roughly fit them into three buckets:
The limitations of the technology
The scarcity of resources
The inevitability of governance