Big model companies "racing" on chips, is Nvidia in danger?
TechFlow Selected TechFlow Selected
Big model companies "racing" on chips, is Nvidia in danger?
The chip war continues among the established chip manufacturers.
Navigating Web3 tides with focused insightsBy Mu Mu
The competition in artificial intelligence isn't just about large models from tech giants like OpenAI and Google. Behind the scenes, the chip赛道 powering these computations is also heating up, with even the makers of large models now entering the race.
OpenAI plans to order more efficient NPU chips from Rain AI, a startup backed by CEO Sam Altman; Microsoft has unveiled two self-developed chips—Azure Maia 100 and Azure Cobalt 100; and Google’s newly launched Gemini 1.0 model runs on its own custom-designed TPUs v4 and v5e chips.
Until recently, Nvidia had built an empire as the dominant supplier of AI chips for these large-model companies. Now, however, those same companies are attempting to achieve partial self-sufficiency. Yet it's still the established players in the semiconductor industry who are formally challenging Nvidia's throne.
Advanced Micro Devices (AMD) has launched its next-generation AI chip, the MI300X, and successfully captured three major customers—Meta, Microsoft, and OpenAI—from Nvidia.
The AI chip race is intensifying, but dethroning Nvidia—the current king—is no easy feat.
Large Model Companies Building Their Own Chips
This year has seen a boom in AI large models and applications, leading to frenzied demand for Nvidia’s A100, A800, H100, and H800 chips capable of training large AI models. The buyers aren’t limited to tech firms—they include governments and venture capital firms alike.
Nvidia, the "water seller" of this gold rush, has enjoyed unprecedented momentum. Its AI chips are in such high demand that another GPU shortage has emerged in the AI market.
Microsoft highlighted concerns in its 2023 fiscal report about securing enough GPUs for its cloud operations. OpenAI CEO Sam Altman has repeatedly voiced public frustration over chip shortages and soaring costs. In May, he stated openly that OpenAI was experiencing a severe computing power shortage, which has begun affecting user experience—with ChatGPT frequently lagging and responding slowly.
Reports suggest that running ChatGPT costs OpenAI around $700,000 per day. According to Reuters, each ChatGPT query costs approximately four cents. If such queries reached one-tenth of Google's search volume, it would require an upfront investment of roughly $48 billion in GPUs, with annual chip expenses amounting to $16 billion just to keep the system running.
With computing shortages and high costs, Peter Marrs, President of Dell Asia Pacific and Japan, predicted that buyers would not tolerate long delivery times for Nvidia GPUs—creating opportunities for numerous competitors.
To reduce reliance on Nvidia and address the global GPU shortage, OpenAI is exploring developing its own AI chips to lower GPT training costs.
Recently, a letter of intent from OpenAI surfaced, revealing that during Altman’s tenure as CEO, the company committed to purchasing chips worth up to $51 million from Rain AI—a startup in which Altman has invested.
Notably, these chips are neuromorphic “brain-inspired” AI NPUs based on technology said to “mimic the structure and function of the human brain,” enabling parallel and distributed information processing ideal for compute-intensive AI tasks, offering low power consumption and high efficiency. However, the chip remains in development.
OpenAI isn’t alone—tech giants like Microsoft and Google have long been building more efficient chips.
On November 16, Microsoft unveiled two self-developed chips at its annual Ignite conference for IT professionals and developers: the cloud-based AI chip Azure Maia 100 and the server CPU Azure Cobalt 100.
The Maia 100 is designed for cloud-based AI training and inference workloads, while the Cobalt 100 targets general-purpose computing. Microsoft’s data centers expect to adopt both Arm CPUs and dedicated AI accelerators by early 2024. Microsoft noted that besides testing the chip within Bing and Office AI products, OpenAI is also evaluating it.
Microsoft develops Maia 100 for internal use and to supply partner OpenAI
Google has also taken action. Its latest large model, Gemini 1.0—claimed to outperform GPT-4—runs on Google’s proprietary TPUs v4 and v5e chips.
Google says Gemini runs significantly faster on TPUs than earlier, smaller, less powerful models. Additionally, Google released the Cloud TPU v5p system, designed to support training cutting-edge AI models and accelerate Gemini’s development.
Hardware makers including Apple and Huawei are increasingly designing and developing their own chips to meet specific business needs and gain competitive differentiation.
Nvidia's Defense and Expansion
Can large model companies truly break free from their dependence on Nvidia by entering the chip赛道?
Despite the fact that Nvidia’s H100 GPU prices have doubled yet remain in short supply, even companies like Google—which already uses self-developed chips—continue to purchase large quantities of Nvidia hardware.
Nvidia has strong moats.
According to the UK’s Financial Times, Nvidia has invested in more than two dozen companies this year—from multibillion-dollar AI platform startups to small ventures applying AI to healthcare or energy sectors.
Although Nvidia claims it doesn’t impose special terms or require portfolio companies to use its chips, such investments naturally foster closer relationships.
Mohamed Siddeek, head of Nvidia’s venture arm NVentures, said: “For Nvidia, the primary criterion for investing in startups is relevance.” He emphasized, “Companies that use our technology, rely on our technology, or build businesses on our platforms—I can’t think of a single company we’ve invested in that doesn’t use Nvidia products.”
Dealroom, a firm tracking venture capital activity, estimates that Nvidia participated in 35 deals in 2023—nearly six times last year’s total. Dealroom calls this Nvidia’s most active year in AI-related investments, surpassing Silicon Valley heavyweights like Andreessen Horowitz and Sequoia Capital.
Moreover, Nvidia’s CUDA computing platform and its broader software-hardware ecosystem form an even sturdier defensive wall.
CUDA is a parallel computing architecture developed by Nvidia. Under equivalent conditions, Nvidia GPUs supporting CUDA can be 10 to 100 times faster than CPUs. It was precisely thanks to CUDA that GPUs surpassed CPUs to become the foundation of modern big data computing.
However, large model companies face significant hurdles when developing their own chips—chief among them being material shortages.
Rob Enderle, principal analyst at The Enderle Group, said, “Making chips is not easy. Foundries and wafer fabs are already at capacity, making it highly likely that OpenAI’s effort could fail.” He added, “They’d be better off partnering with AMD, Qualcomm, Nvidia, or Intel, all of whom already own foundries.”
Then there’s cost.
Todd R. Weiss, senior analyst at Futurum Group, said creating their own chips may seem like a “cool idea at first glance,” but the ongoing costs of chip design, building fabrication facilities, continuously developing updated roadmaps for better chips, and managing supply chains mean “it’s not simpler than buying chips from others.”
The real battlefield remains between chip manufacturers.
On December 6, AMD—the biggest rival to Nvidia—held its “Advancing AI” launch event, bringing executives from Microsoft, Meta, and other tech firms onstage. At the event, AMD unveiled its new AI chip, the MI300X.
Compared to Nvidia’s H100 HGX, the MI300X accelerator delivers significantly better throughput and latency performance in large language model inference, at a lower price point. Following the investor event, Meta, Microsoft, and OpenAI announced they would begin using AMD’s latest AI chips.
Real cost reduction will ultimately come from competition between chipmakers. Only when giants like AMD and Nvidia enter a production race will prices truly drop. For now, large model companies developing their own chips merely adds another leg to the arms race.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
