A new AI model called DeepSeek is redefining the approach to artificial intelligence by focusing on efficiency and using lean data. The company’s AI model performs as well as leading US models but at a fraction of the training cost. This approach could accelerate the proliferation of AI startups focused on “small is beautiful” models, leading away from the traditional “bigger is better” paradigm.
The DeepSeek team used only 800,000 examples to transform large language models into reasoning models, demonstrating data efficiency. A Hong Kong University of Science and Technology team replicated the model with just 8,000 examples. This marks the beginning of a new AI race – the Small Data competition.
DeepSeek engineers highlight the challenge of improving reasoning performance with high-quality data as a “cold start.” They devised multiple stages to generate, collect, and fine-tune relevant data. Human ingenuity played a key role in this process, rather than automation.
The reason DeepSeek’s innovation is underhyped is due to the “Moore’s Law addiction” prevalent among US Big Tech companies. This mindset prioritizes bigger models with more data running on the latest processors, ignoring efficiency and smaller models. Nvidia, a pioneer of specialized chips for data processing, has contributed to this paradigm.
However, last year saw some movement away from the “bigger is better” approach. Startups have presented alternatives using smaller models and less data. Even Nvidia has been exploring edge computing and bringing its chips to developers’ desktops.
DeepSeek’s success could accelerate this trend towards “small is beautiful.” As Gil Press, a Forbes contributor, notes, the new paradigm may become an addiction: smaller models or more elaborate models, all utilizing Small Data.
Source: https://www.forbes.com/sites/gilpress/2025/01/30/deepseek-means-the-end-of-big-data-not-the-end-of-nvidia