Nvidia has announced a new family of foundational AI models called Cosmos, designed to train humanoids, industrial robots, and self-driving cars. Unlike language models that focus on text generation, Cosmos focuses on generating images and 3D models of the physical world. The company’s CEO Jensen Huang demonstrated Cosmos’ capabilities at CES, showcasing its ability to simulate warehouse activities.
Cosmos was trained on 20 million hours of real-world footage, including humans walking and manipulating objects. Researchers and startups hope that these foundational models will enhance robots’ capabilities in factories and homes. Users can fine-tune the models using their own data, allowing for more realistic training scenarios.
Several companies are already using Cosmos, including humanoid robot startups Agility and Figure AI, as well as self-driving car companies like Uber, Waabi, and Wayve. Nvidia also announced software to help robots learn new tasks more efficiently, complementing its existing Isaac platform.
Additionally, Nvidia unveiled Project Digits, a $3,000 “personal AI supercomputer” that can run large language models without cloud services. The company also announced next-generation RTX Blackwell GPUs and incoming software tools for building AI agents.
Source: https://www.wired.com/story/nvidia-cosmos-ai-helps-robots-self-driving-cars