The era of the “gold rush” in artificial intelligence (AI) is coming to an end. Despite significant investments from tech giants like Microsoft, Alphabet, Amazon, Meta Platforms, and Nvidia, the field’s growth has slowed due to a shortage of novel data for training software.
Researchers had long believed that bigger systems would produce better results, leading to an “arms race” for chips, data centers, and computing power. However, recent advancements have shown that evidence for the AI scaling law is unraveling. Cutting-edge systems have consumed most available training data, and multiple labs have struggled to achieve significant improvements in next-generation models.
Some researchers hope that future advances will come from better algorithms rather than brute-force approaches. Techniques like “test-time compute” aim to enhance the inference process, allowing models to spot patterns or use new data more efficiently. However, this approach is less ambitious than the exponential improvement envisioned by AI proponents.
As a result, companies like Nvidia and OpenAI may face challenges. Nvidia’s silicon business has benefited from the rush for access to its chips, but the end of AI scaling could lead to decreased demand. Model developers like OpenAI might also feel ambivalent, as their financial statements will benefit from reduced training costs, but the loss of the scalability advantage undermines part of their bull case.
The behemoths in the industry, including Microsoft and Alphabet, may now face less existential risk, as they are no longer competing for a superintelligent giant model that could perform any task. This shift allows them to reassess their spending priorities and wait for more revenue to justify their investments.
However, an end to the capex arms race may also mean lower barriers to entry for new startups, who could produce competitive AI products at minimal cost by leveraging open-source models or modifying widely available systems. A new wave of enterprise-software businesses might emerge, tailored to specific industries like law firms or coders.
Ultimately, if AI training costs stop spiraling upwards, it’s likely welcome news for investors overall. This would follow a recent decline in the price of inference, making AI adoption more feasible and allowing early signs of progress to proliferate.
Source: https://www.reuters.com/breakingviews/ai-models-slowdown-spells-end-gold-rush-era-2024-12-12