OpenAI reportedly builds custom AI chips as it embraces AMD — company also abandons plans to build its own fabs
Broadcom set to help OpenAI to build its AI inference chip.
OpenAI is proceeding with the development of its first custom AI chip with Broadcom and expects to produce it for TSMC. However, reports Reuters, the company no longer intends to spearhead building a fab network. At the same time, the company continues to add more powerful chips from AMD and Nvidia to its fleet.
AI inference chips incoming, no more fab plans
In a bid to reduce reliance on Nvidia, OpenAI initially considered developing its chips both for training and inference and then facilitating the building of a dozen fabs (operated by prominent foundries like TSMC and Samsung Foundry), but high costs and long timelines made it impractical. Instead, OpenAI has prioritized designing custom AI chips for inference together with Broadcom and producing them at TSMC. For now, OpenAI will keep using GPUs from Nvidia and AMD for training.
While high-demand AI GPUs like Nvidia's H100 and H200 are used for training of large language modes by pretty much everyone, which is why they are hard to get, demand for AI inference chips is projected to grow as more AI applications reach the market. OpenAI's upcoming custom-designed inference chip is slated for release by 2026. According to Reuters, this timeline could be adjusted based on project needs, but the focus is on inference tasks that enhance real-time AI responses.
To support this new chip development, OpenAI has assembled a team of around 20 engineers led by experienced engineers like Thomas Norrie and Richard Ho, specialists who previously worked on Google's Tensor Processing Units (TPUs). The team is key to moving forward with the in-house design, which could allow for greater customization and efficiency.
OpenAI now does the same thing as Amazon Web Services, Google, Meta, and Microsoft. These companies have chips for AI or general-purpose workloads, sometimes co-developed with Broadcom.