A normie's guide to AI hype
A normie's guide to AI hype
What the excitement is about, how it came to be and what might slow it down
TL;DR
This newsletter is about the current AI hype cycle. It looks at what the excitement is about, how it came about, what it is being driven by and what might slow it down.
Here are the key takeaways:
The AI hype is only interesting if you think AI is important. If AI will impact your life or work somehow, then understanding its current frenzy may help determine where it could be going next.
The AI hype represents the latest ideas about what AI is and what AI could be. It is about both what current models are capable of and what they could be capable of in the future if their limitations are overcome.
The current AI hype owes to several important landmarks in the development of generative AI. This includes the various models released in recent years by OpenAI and others.
Two prospects drive the current hype:
Generative AI will significantly improve our society by enabling unparalleled increases in productivity and growth.
Generative AI, and LLMs in particular, constitute AGI or at least puts us on the path to AGI and beyond.
Like any hype cycle, there are ups and downs. Things that could cool down the excitement around AI include:
The maturation of generative AI
AI development hitting a wall
Semiconductor supply chain issues
Regulation
Should you care about the AI hype?
The AI hype is only interesting if you think AI is important.
You might think AI is important for various reasons, especially if you believe it will somehow have an impact on your life or work. If so, then understanding the current frenzy around AI may help determine where it could be going next.
This is regardless of whether you think the frenzy is warranted or not. I have tried to stay neutral on this point, and so exactly how seriously you should take the hype is up to you.
What is the AI hype?
The AI hype represents the latest ideas about what AI is and what AI could be.
Regarding what AI is, this is about what the current models are capable of doing as well as their limitations. AI development is focused on creating entities that can carry out tasks that humans can do, and so the capabilities of these models are ultimately determined by how well it can do those tasks.
This then leads to speculation about what AI could be. We use the capabilities of today’s models as a starting point to think about whether their limitations can be overcome and the capabilities this could lead to.
Google recently released Gemini, its most advanced LLM yet. According to Google, the model performs better than OpenAI’s GPT-4 on several benchmarks testing for various capabilities including multi-step reasoning, basic arithmetic manipulations and code generation.
This means that models like Gemini and GPT-4 could be useful for a range of different tasks. This includes writing essays, augmenting search engines or functioning as programming assistants.
There is already some proof of these models actually being capable of these tasks. Some research has found that since ChatGPT’s release, the demand for copywriters and graphic designers has fallen significantly.
https://x.com/stefanfschubert/status/1722910065378668646
But some argue that AI may reach even more impressive capabilities. As these models improve, they can be relied on for tasks of increasing complexity and value.
The excitement around this determines the intensity of the hype. There are therefore times of high excitement and times of low excitement, which can be thought of as AI summers and AI winters respectively.
An AI summer is when there are high expectations about the current and future capabilities for AI. This usually follows some significant event in its development, such as the release of ChatGPT.
Contrastingly, an AI winter is when there are low expectations about the current and future capabilities for AI. This happens when promised breakthroughs fail to materialise or end up being quite unspectacular.
There are different indicators for determining what stage of the AI hype cycle we are experiencing. This could be the level of investment, model usage or the financial performance of companies producing chips for AI development and deployment.
These indicators currently suggest that we might be in an AI summer. Over $17 billion has been pumped into AI startups, AI usage worldwide per month has reached over 500 million and Nvidia's share price has appreciated over 200% after the launch of ChatGPT.
These facts among others demonstrate much excitement about what AI is currently capable of and what it might be capable of in the future. This excitement has been growing for several years now.
How did the AI hype come about?
The current AI hype owes to several important landmarks in the development of generative AI.
2012 is a good place to start, which is when AI researchers Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton released an image classifier using neural networks called AlexNet. This model performed significantly better than others and demonstrated the usefulness of deep learning for computer vision, speech recognition and other purposes.
But the current hype did not really start until ten years later. It was during 2022 that the excitement around generative AI models started to spread beyond the research labs.
In June 2022, Google engineer Blake Lemoine claimed that Google's language model was sentient. Despite his claims being dismissed by the company, Lemoine went public with his discovery and published the conversations he had with the model.
Numerous generative AI models were also released that year. This included the image generation models like Midjourney and DALL•E 2 as well as the famous text generation model ChatGPT.
The hype continued in 2023. Microsoft invested $10 billion into OpenAI, GPT-4 was released, and more capable open-source models began to proliferate.
Generative AI is now central to the current hype. These models have led the conversation around what AI is and what AI could be.
What is driving the current AI hype?
Two prospects drive the current AI hype:
Generative AI will significantly improve our society by enabling unparalleled increases in productivity and growth.
Generative AI, and LLMs in particular, constitute AGI or at least puts us on the path to AGI and beyond.
The first prospect relates to effective accelerationism, or e/acc for short. In the context of AI, e/acc proposes that the solution to humanity’s problems lies in technological advancement.
A big proponent of e/acc is the venture capitalist Marc Andreessen. He published an extensive article explaining how AI will augment human intelligence and lead to new medicines, solutions to climate change and a host of other improvements to our world.
Accordingly, generative AI is viewed by some as a tool for increasing our productivity and advancing, or accelerating, humanity forward. This is partly why we see a push for AI to be in all sorts of products and services that we use everyday.
This leads to the second prospect, which is about AGI. This is essentially concerned with creating AI that is as capable as humans.
There is no universal agreement on how we identify when AGI has been achieved, though different ideas have been suggested. OpenAI for instance, which has made building AGI a core part of its mission, defines it as “highly autonomous systems that outperform humans at most economically valuable work.”
The most advanced LLMs to date have shown themselves to be capable of many impressive tasks, but there remains doubt about whether they constitute AGI. Research has shown that GPT-4, for instance, is not actually capable of generalising beyond its training data and is also poor at abstraction and reasoning tasks compared to humans.
Nevertheless, some believe that models like GPT-4 might put us on the path to AGI. Sam Altman, OpenAI CEO, believes that LLMs are just one component of a wider system that will be capable of completing the tasks thought to be exclusively reserved for humans.
The prospect of AGI fuels the hype because, according to e/acc, it could lead to massive growth that benefits our society. But a particular aspect of this that fuels the hype further is the possibility of achieving artificial superintelligence (ASI), entities that possess an intelligence exceeding that of humans.
Some think the timeline for this could be very quick. Nick Bostrom, author of Superintelligence: Paths, Dangers, Strategies, believes that while it might take a while to reach AGI, once this happens the ascent to ASI “will be very rapid.”
But ASI makes many people fearful. One such group are the effective altruists.
Effective altruism (EA) is about supporting effective ways to improve society and alleviate suffering. According to this movement, one of the most important problems facing humanity is the existential risk of ASI.
This is how Bostrom explains the existential risk in Superintelligence:
What makes humans different from other species is our minds. Our general intelligence has enabled various improvements to society through language, technology and complex organisations.
If we manage to build ASI, our fate may become dependent on such entities as we cease to be the most significant agents of change on this planet. We might end up creating something more powerful than us and that will also have power over us.
However, we are still in control of whether this happens since we would need to build the superintelligent entity in the first place. It is therefore imperative that we develop ASI in a way that prevents it from pursuing objectives inconsistent with humanity's interests.
The prospects driving the current AI hype cycle are manifestations of what AI is and what AI could be. The first prospect is mainly concerned with what AI is, and the second prospect is mainly concerned with what AI could be.
There are different views on these two prospects. Some, like the accelerationists, are excited about what they could entail, while others, like the altruists, are more cautious.
These perceptions ultimately fuel the AI hype. They entertain the idea that the latest AI models possess, or will eventually possess, capabilities that could be to our benefit or detriment.
How does the hype slow down?
Like any hype cycle, there are ups and downs. The current hype around AI will therefore not continue forever.
In my view, there are five things that could cool down the excitement around AI:
The maturation of generative AI
AI development hitting a wall
Semiconductor supply chain issues
Regulation
The maturation of generative AI
We will eventually get a better sense of what generative AI can and cannot be used for.
When this happens, the best use cases for these models will become clearer and other more speculative uses will be attempted less. In turn, the excitement around AI may lessen as we concentrate on more proven uses.
This maturation may also happen as companies realise that trying to disrupt certain industries with generative AI may not be so straightforward. Certain sectors, such as legal, are protected by a structural rigidity that may permit increased uses of AI but not replacement by AI.
Erik Hoel calls this 'human-preferenced bottlenecks' and it is a factor often missed in the discussions about AI. Just because a machine can do something a human can do does not mean it inevitably will, for there are social frictions that its creators must also overcome.
AI development hitting a wall
Algorithms, data and computing power are three crucial elements that can hinder, as well as enable, progress in AI.
The improvement of these elements is what has given rise to the generative AI models of today. We have better algorithms, more data and more powerful computer chips than ever before.
However, whilst these elements are enablers of AI, they are also limitations on what AI is and what AI could be. AI is only as good as the algorithms it executes, the data its trained on and the computing power available.
As mentioned before, the most advanced LLMs still fall short of some of the capabilities associated with general intelligence. This includes things like common sense, cumulative learning, planning and efficient management of computational activity.
Deep learning, as powerful as it is for next-word prediction, it is yet to prove sufficiently reliable for more general applications. This approach is limited to solving the task of mapping specific inputs and outputs together only on the instruction of a human programmer.
Better algorithms may therefore be needed to achieve what some think AI could be. The longer it takes to build these improved algorithms, the more time the hype has to cool down.
There are some who argue that the key to better models is making them bigger, but this is subject to the limitations of computing hardware. Increasing the size of models like ChatGPT, which already have billions of parameters, may prove impractical for the current stack of computing chips available.
Additionally, generative AI needs sufficiently high-quality training data, of which could soon be running out according to some estimates. This is why OpenAI for example is seeking partnerships with publishers and other organisations to collect the data needed for its next models.
Semiconductor supply chain issues
The supply chain for computer chips involves numerous companies across the world. No one country completely dominates the industry, which makes it quite complex and more vulnerable to disruption.
This was especially highlighted during Covid-19. National lockdowns caused a fall in demand for automobiles but an increase in demand for consumer electronic products like laptops, tablets and gaming consoles.
This caused production for automobiles to be cut while manufacturers pivoted to sectors with growing demand. The uncertainty also caused the hoarding of semiconductors by some companies, contributing to an under-supply of chips.
When the pandemic began to abate and economies opened back up, demand for automobiles began to rise as consumers returned to the market. However, having focused much of their production in certain areas, chipmakers were generally slow to react which caused a scarce supply of semiconductors.
As such, the semiconductor supply chain is a sluggish one that usually struggles to respond quickly to changes in demand or the impact of external events. AI developers therefore rely on a tightly coupled system that could easily impede their work.
Regulation
Data rights law will impose limits on the development and deployment of AI.
For example, data protection rules require developers to justify the collection and use of data for training their AI models. This could be especially difficult when such data is scraped from the internet, which remains standard for building the training datasets for generative AI.
The EU's AI Act will also play a role. This will impose various rules around consumer safety, transparency, risk management and accountability for developers and users of AI systems.
Accordingly, as AI becomes more capable and widespread, there will be greater efforts to shape how it pans out. The work on AI governance, ethics and compliance will certainly increase.
This will add to the workload for developers and also stem the excitement around what AI is and what AI could be. Instead of moving fast and breaking things, AI development will become more about slowing down and fixing things.