Part 6/8:
A noteworthy aspect of Grock's development is the transparency regarding its training infrastructure. The modeling was accomplished on the Colossus supercomputer located in Memphis, Tennessee, which is recognized as the world's largest AI supercomputer. This powerhouse consists of over 200,000 Nvidia H100 GPUs, with ambitious plans to expand its capacity to 1 million GPUs. The supercomputer's enormous energy demands have prompted the installation of portable diesel generators to supplement grid electricity.