Part 3/8:
In discussing the evolution of Gro, the engineering team highlighted their rapid advancement over the last 17 months. Beginning with Grok 1, which had only 314 billion parameters, the team has now reached unprecedented heights with Grok 3, leveraging a massive GPU training cluster to drive performance. They recounted the challenges faced earlier in their development, including power and cooling issues while trying to train models at scale.
To address their growing needs, a critical decision was made to build their own data center, resulting in the world's largest fully connected accelerator cluster in just 122 days. This monumental feat required a considerable amount of coordination and innovation, as well as extensive infrastructure setup, including reliable power and cooling systems.