What Exactly Is the "Brain-Like" AI Chip That OpenAI Is Heavily Betting On?
TechFlow Selected TechFlow Selected
What Exactly Is the "Brain-Like" AI Chip That OpenAI Is Heavily Betting On?
Derived from RISC-V, it can improve inference and training energy efficiency by 10,000 times.
Navigating Web3 tides with focused insightsAuthor: Mei Yi, GeekPark
Just as OpenAI's power struggle came to an end, a key deal quietly surfaced.
According to reports from Wired, during Sam Altman’s tenure as CEO of OpenAI, the company signed a $51 million letter of intent with Rain AI, committing to purchase chips once they become available.
Rain AI is an AI chip startup aiming to drastically reduce the cost of AI computing. By developing an AI chip—NPU—that mimics the way the human brain works, it intends to provide companies like OpenAI and Anthropic with “low-cost, high-efficiency hardware.”
The company claims, “Compared to traditional GPUs, NPUs will offer AI developers (such as OpenAI) up to 100x computational capability and 10,000x energy efficiency improvements in training.”
Given that OpenAI has long struggled with computing shortages, it’s no surprise that it would invest heavily to secure chip supplies for its AI projects.
What makes Rain AI’s chips special? How did this company emerge into prominence? And what does this investment reveal about Altman and OpenAI’s strategy in the semiconductor space?
01 "Brain-Inspired" AI Chips
Rain AI’s core product is a “brain-inspired” AI chip—the NPU—based on neuromorphic technology. The chip aims to process information efficiently and with low power consumption, meeting the demanding computational needs of AI tasks.
It mimics the structure and function of the human brain, building networks of artificial synapses similar to neural connections in the brain. This architecture allows the NPU to process information in parallel and distributed ways, making it ideal for compute-intensive tasks in AI applications.
Moreover, Rain AI was among the first to adopt digital in-memory computing (D-IMC), further improving efficiency in AI processing, data movement, and storage.
Rain AI’s NPU aims to efficiently process information with low power consumption, addressing the rigorous computing demands of AI tasks | Image source: Rain AI official website
In addition, Rain offers IP licensing opportunities for its D-IMC tiles and software stack—specifically tailored for AI workloads requiring ultra-low latency and high energy efficiency on devices. These cover a range of computing use cases via Long Reach Ethernet (LRE), including smart cars and smartwatches.
For its own products, Rain uses the slogan “Redefining the Limits of AI Computing,” promoting that “our AI accelerators achieve an unprecedented balance between speed, power, area, accuracy, and cost.”
Given that Rain’s designed “brain-inspired” chip (NPU) promises efficient and low-power operation, this is critical for overcoming bottlenecks associated with heavy-duty chips made by companies like NVIDIA and AMD.
Covering a series of LRE computing use cases | Image source: Rain AI official website
Gordon Wilson, one of Rain AI’s co-founders, boldly stated on LinkedIn: “NPU chips will define a new AI chip market and dramatically disrupt the existing one.”
However, it’s worth noting that while Rain AI claims better efficiency than NVIDIA’s GPUs, its initial chip is actually based on the traditional open-source RISC-V architecture supported by Google, Qualcomm, and other tech firms—designed for so-called edge devices away from data centers, such as phones, drones, cars, and robots.
Currently, most edge chip designs, such as those used in smartphones, focus primarily on the inference phase of neural networks. Rain, however, aims to deliver a chip capable of both model and algorithm training and subsequent inference operations.
To date, Rain AI has launched its first AI platform capable of both AI inference and training, claiming that its “brain-inspired” chip (NPU) will allow AI models to be customized or fine-tuned in real time based on their environment.
Sam Altman has publicly stated, “This neuromorphic approach could significantly reduce the cost of AI development and may help achieve true AGI.”
Reportedly, OpenAI hopes to use these chips to lower data center costs and deploy its models directly onto devices like phones and watches—making the “brain-inspired” chip (NPU) undeniably attractive to OpenAI.
Nevertheless, all of this remains speculative. How OpenAI will actually use Rain’s chips remains unknown for now.
02 Closely Tied to OpenAI
Founded in 2017, Rain AI aims to build “low-cost” computing platforms for future AI.
Rain AI has three co-founders—Jack Kendall, Gordon Wilson, and Juan Claudio Nino—who met at the University of Florida. The company also hired Scott Gray, a former OpenAI software engineer, as an advisor.
Currently, Rain AI employs around 40 people, including experts in AI algorithm development and traditional chip design.
Rain AI’s founding team | Rain AI
Interestingly, Rain AI’s headquarters is located in San Francisco, less than a mile away from OpenAI.
The year after its founding, Rain AI raised $5 million in seed funding, with investors including the renowned startup accelerator Y Combinator.
At the time, Altman was CEO of Y Combinator and personally invested $1 million in Rain AI. A year later, OpenAI approved the $51 million chip purchase agreement.
By April 2022, following a $25 million funding round led by Saudi Arabia’s Prosperity7 Ventures, Rain had raised a total of $33 million, reaching a valuation of $90 million.
Earlier this year, the company boasted to potential investors about its progress, saying it expected to release a “test” chip that month—indicating that chip design was complete and manufacturing could begin.
Rain AI also said it could deliver its first chips to customers as early as October next year, emphasizing to investors that it had held advanced talks with tech giants including Google, Oracle, Meta, Microsoft, and Amazon to sell systems to them. Microsoft declined to comment, while the others did not respond to requests for comment.
In short, Rain AI remains in development, and it’s unclear when its chips will be commercially available. While the company’s “brain-inspired” chip (NPU) technology holds great promise and has high-profile supporters, it still faces many challenges.
03 OpenAI’s Ambitions
Regardless of whether Altman’s investment in Rain AI involved personal motives, chip shortages are indeed a major challenge OpenAI faces.
In fact, just a week after ChatGPT’s launch a year ago, Altman already found computing costs “brutal.” Since then, he has repeatedly complained publicly about the “brutal scarcity” and “staggering” costs of AI chips.
In May this year, Altman reluctantly admitted, “OpenAI is experiencing severe computing shortages, causing delays in many short-term plans.”
As everyone knows, OpenAI relies heavily on Microsoft’s powerful cloud services as its primary investor. Yet due to hardware limitations, it frequently disables certain ChatGPT features.
Altman noted, “The pace of AI advancement may depend on new chip designs and supply chains.” After all, computing power is everything today.
Indeed, Altman himself began investing in chip ventures years ago. Besides Rain AI, around 2021 he also invested in Cerebras—the AI company known for its wafer-scale chip, large as a dinner plate, requiring two hands to hold.
Chip as large as a plate | Image source: Cerebras official website
Earlier this year, when “silicon sage” Jim Keller and “silicon prodigy” Sam Zeloof founded Atomic Semi—a company aiming to rapidly produce affordable chips by simplifying and miniaturizing semiconductor fabs and circuit prototypes—Altman took notice, and OpenAI Startup Fund participated in the investment.
Image source: Analytics India Magazine
Additionally, just weeks before Altman was ousted from OpenAI, news emerged that he was attempting to raise billions of dollars to start a new chip company.
Details of the project remain unknown, except that it was codenamed “Tigris,” aiming to compete with NVIDIA in the AI chip domain.
Reportedly, Altman sought funding for the “Tigris” project in the Middle East. The geographical coincidence raises speculation about possible links between this project and Rain AI.
Furthermore, Altman reportedly held discussions with semiconductor executives, including those from chip design firm Arm, exploring how to design new chips earlier to reduce costs for large language model companies like OpenAI.
And it’s not just Altman—OpenAI itself is actively seeking cheaper ways to build large models, aiming to reduce dependence on NVIDIA.
Beyond investing in chip suppliers like Rain AI, OpenAI has recently begun exploring in-house chip development, evaluating potential acquisition targets, and hiring personnel for hardware-related roles.
Image source: OpenAI official website
Recently, OpenAI appointed Richard Ho, former head of Google’s TPU, as its hardware lead, hired several compiler and kernel experts, and is currently recruiting a “data center facilities design expert.”
Richard Ho will lead OpenAI’s new hardware division and help optimize partner data center networks, racks, and infrastructure.
However, these forward-looking investments still fall short of solving the immediate GPU shortage. Currently, OpenAI continues to rely heavily on NVIDIA chips.
Observations show that OpenAI is dynamically adjusting the capabilities of products like ChatGPT to save computing power. This explains why some users have recently noticed GPT-4 appearing lazier than GPT-3.5.
With the rise of large models, attention has turned to the power consumption of massive AI model data centers. Companies like Rain and other chip startups aim to reconfigure data processing methods to reduce transmission needs and lower power usage.
Google, Microsoft, AMD, Intel, Amazon, along with startups like Cerebras, Sambanova, and Rain, are all entering the future of AI chips. Will the AI computing supply market shift? Can OpenAI break free from reliance on external computing power? Given the long cycles involved in chip development, these challenges are likely to persist for quite some time.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
极客公园
