Elon Musk has started training the “world’s most powerful AI” using a massive cluster of 100,000 Nvidia H100 GPUs. The Memphis Supercluster began training at 4:20am local time and is expected to be completed by December this year.
Source: https://www.tomshardware.com/pc-components/gpus/elon-musk-fires-up-the-most-powerful-ai-training-cluster-in-the-world-uses-100000-nvidia-h100-gpus-on-a-single-fabric