IOSG Research | GPU Supply Crisis: How Early-Stage AI Startups Can Break Through
TechFlow Selected TechFlow Selected
IOSG Research | GPU Supply Crisis: How Early-Stage AI Startups Can Break Through
The DePIN model has the potential to address GPU availability issues, but a fragmented model won't make the situation better.
Navigating Web3 tides with focused insightsAuthor: Mohit Pandit, IOSG Ventures
Abstract
-
GPU shortage is real—supply and demand are tight—but the number of underutilized GPUs could meet today’s supply constraints.
-
An incentive layer is needed to encourage participation in cloud computing and ultimately coordinate compute tasks for inference or training. The DePIN model fits perfectly here.
-
Supply-side incentives make it attractive for the demand side due to lower compute costs.
-
Not everything is ideal—choosing Web3 cloud involves trade-offs such as 'latency.' Compared to traditional GPU clouds, trade-offs also include insurance and service level agreements (SLAs).
-
The DePIN model has the potential to solve GPU availability issues, but a fragmented model won’t improve the situation. For exponentially growing demand, fragmented supply is no better than no supply at all.
-
Given the number of new market entrants, market aggregation is inevitable.
Introduction
We stand on the brink of a new era in machine learning and artificial intelligence. While AI has existed in various forms for some time (AI being computers instructed to perform human-doable tasks, like washing machines), we are now witnessing the emergence of sophisticated cognitive models capable of executing tasks requiring intelligent human behavior. Notable examples include OpenAI's GPT-4 and DALL-E 2, as well as Google's Gemini.
In the rapidly expanding field of artificial intelligence (AI), we must recognize two aspects of development: model training and inference. Inference encompasses the functionality and output of AI models, while training includes the complex processes required to build intelligent models (including machine learning algorithms, datasets, and computational power).
Take GPT-4 as an example—the end user only cares about inference: obtaining output from the model based on textual input. However, the quality of this inference depends on model training. To train effective AI models, developers need access to comprehensive foundational datasets and massive computational power. These resources are primarily concentrated in industry giants including OpenAI, Google, Microsoft, and AWS.
The formula is simple: Better model training >> leads to enhanced inference capabilities of AI models >> attracting more users >> generating more revenue, which fuels further investment in training resources.
These major players have access to large foundational datasets and, more critically, control vast amounts of computational power, creating entry barriers for emerging developers. As a result, newcomers often struggle to obtain sufficient data or leverage necessary computational power at economically viable scale and cost. Given this context, networks hold significant value in democratizing resource access, particularly concerning large-scale acquisition of computational resources and reducing costs.
The GPU Supply Problem
NVIDIA CEO Jensen Huang declared "Moore's Law is over" at CES 2019. Today's GPUs are highly underutilized. Even during deep learning/training cycles, GPUs remain underused.
Below are typical GPU utilization figures across different workloads:
-
Idle (just booted into Windows OS): 0–2%
-
General productivity tasks (writing, light browsing): 0–15%
-
Video playback: 15–35%
-
PC gaming: 25–95%
-
Active graphic design/photo editing workloads (Photoshop, Illustrator): 15–55%
-
Video editing (active): 15–55%
-
Video editing (rendering): 33–100%
-
3D rendering (CUDA/OptiX): 33–100% (often misreported by Windows Task Manager—use GPU-Z)
Most consumer devices with GPUs fall into the first three categories.
GPU runtime utilization %. Source: Weights and Biases
The above highlights a problem: poor utilization of computational resources.
There is a need to better utilize consumer GPU capacity—even during peak GPU utilization periods, efficiency remains suboptimal. This points to two key future directions:
-
Resource (GPU) aggregation
-
Parallelization of training tasks
Regarding types of hardware available for supply, there are currently four types:
· Data center GPUs (e.g., Nvidia A100s)
· Consumer GPUs (e.g., Nvidia RTX3060)
· Custom ASICs (e.g., Coreweave IPU)
· Consumer SoCs (e.g., Apple M2)
Aside from ASICs (which are purpose-built), the other hardware types can be pooled for optimal use. With many such chips held by consumers and data centers, a DePIN model aggregating suppliers may be a viable path forward.
GPU production follows a volume pyramid; consumer-grade GPUs are produced in the highest volumes, while high-end GPUs like NVIDIA A100s and H100s are produced in the lowest quantities (but offer higher performance). Producing these advanced chips costs 15 times more than consumer GPUs, yet they don’t always deliver 15 times the performance.
The entire cloud computing market is valued at approximately $483 billion today and is expected to grow at a CAGR of around 27% in the coming years. By 2023, ML compute demand will reach roughly 13 billion hours, equating to about $56 billion in spending on ML compute at current standard rates. This entire market is also growing rapidly—doubling every three months.
GPU Demand
Compute demand primarily comes from AI developers (researchers and engineers). Their main needs are: price (low-cost compute), scale (large amounts of GPU compute), and user experience (ease of access and usability). Over the past two years, GPU demand has surged due to increased demand for AI-based applications and advancements in ML models. Developing and running ML models requires:
-
Massive compute (from accessing multiple GPUs or data centers)
-
Ability to execute model training, fine-tuning, and inference, with each task deployed across large numbers of GPUs running in parallel
Spending on compute-related hardware is projected to grow from $17 billion in 2021 to $285 billion in 2025 (about 102% CAGR), and ARK Research forecasts it will reach $1.7 trillion by 2030 (43% CAGR).
ARK Research
With numerous LLMs in the innovation phase, competition driving demand for more parameters and retraining, we can expect sustained demand for high-quality compute in the coming years.
Where Does Blockchain Come In Amid New GPU Supply Constraints?
DePIN models come into play when resources are underutilized:
-
Bootstrap supply side, creating abundant supply
-
Coordinate and complete tasks
-
Ensure tasks are correctly completed
-
Properly reward providers who complete the work
Aggregating any type of GPU (consumer, enterprise, high-performance, etc.) may pose utilization challenges. When compute tasks are split, A100 chips should not handle simple computations. GPU networks need to decide which types of GPUs to include in their network based on their go-to-market strategy.
When computational resources themselves are dispersed (sometimes globally), choices must be made by users or the protocol itself regarding which compute framework to use. Providers like io.net allow users to choose among three compute frameworks: Ray, Mega-Ray, or deploying Kubernetes clusters to execute compute tasks within containers. There are other distributed computing frameworks such as Apache Spark, but Ray is the most widely used. Once selected GPUs complete their compute tasks, outputs are reconstructed to yield trained models.
A well-designed token model can subsidize compute costs for GPU providers, making such schemes highly attractive to many developers (demand side). Distributed computing systems inherently suffer from latency due to task decomposition and output reconstruction. Thus, developers must balance cost-efficiency against time-to-completion when training models.
Do Distributed Compute Systems Need Their Own Chain?
Networks operate in two ways:
-
Pay per task (or compute cycle) or pay per usage
-
Pay per time unit
In the first approach, a proof-of-work chain similar to what Gensyn attempts can be built, where different GPUs share the “work” and are rewarded accordingly. For a more trustless model, they introduce verifiers and whistleblowers who are rewarded for maintaining system integrity based on proofs generated by solvers.
Another proof-of-work system is Exabits, which does not split tasks but treats its entire GPU network as a single supercomputer. This model appears better suited for large LLMs.
Akash Network added GPU support and began aggregating GPUs into this space. They have an underlying L1 to reach consensus on state (showing work done by GPU providers), a marketplace layer, and container orchestration systems like Kubernetes or Docker Swarm to manage deployment and scaling of user applications.
For a truly trustless system, a proof-of-work chain model would be most effective, ensuring protocol coordination and integrity.
On the other hand, systems like io.net do not build themselves as chains. They choose to focus on solving core GPU availability problems and charge customers per time unit (per hour). They don’t require a verifiability layer because they essentially “rent out” GPUs for unrestricted use during specific lease periods. The protocol itself doesn’t handle task splitting—this is left to developers using open-source frameworks like Ray, Mega-Ray, or Kubernetes.
Web2 vs Web3 GPU Cloud
There are many participants in the Web2 space for GPU cloud or GPU-as-a-service. Key players in this domain include AWS, CoreWeave, PaperSpace, Jarvis Labs, Lambda Labs, Google Cloud, Microsoft Azure, and OVH Cloud.
This follows a traditional cloud business model where customers rent GPUs (or multiple GPUs) per time unit (typically hourly) when they need compute. Numerous solutions exist for different use cases.
The main differences between Web2 and Web3 GPU clouds lie in the following parameters:
1. Cloud Setup Cost
Due to token incentives, the cost of setting up a GPU cloud is significantly reduced. OpenAI is raising $1 trillion for chip production. It seems that defeating market leaders without token incentives would require at least $1 trillion.
2. Compute Latency
Non-Web3 GPU clouds will be faster since rented GPU clusters are geographically localized, whereas Web3 models may involve more widely distributed systems where latency arises from inefficient task splitting, load balancing, and most importantly, bandwidth.
3. Compute Cost
Due to token incentives, Web3 compute costs will be significantly lower than existing Web2 models.
Cost comparison:
These figures may change as more supply becomes available and cluster utilization improves. Gensyn claims to offer A100s (and equivalents) for as low as $0.55 per hour; Exabits promises a similar cost-saving structure.
4. Compliance
Compliance is not straightforward in permissionless systems. However, Web3 systems like io.net and Gensyn do not position themselves as fully permissionless. They address compliance concerns such as GDPR and HIPAA during GPU onboarding, data loading, data sharing, and result sharing stages.
Ecosystem
Gensyn, io.net, Exabits, Akash
Risks
1. Demand Risk
I believe top-tier LLM players will either continue accumulating GPUs or rely on GPU clusters like NVIDIA’s Selene supercomputer, which delivers peak performance of 2.8 exaFLOP/s. They won’t depend on consumer or long-tail cloud providers aggregating GPUs. Currently, leading AI organizations compete more on quality than cost.
For non-heavyweight ML models, however, they will seek cheaper compute resources. Blockchain-based, token-incentivized GPU clusters can serve these needs by optimizing existing GPUs (under the assumption that such organizations prefer training their own models rather than using LLMs).
2. Supply Risk
With massive capital flowing into ASIC research and inventions like Tensor Processing Units (TPUs), the GPU supply issue might resolve itself. If these ASICs offer favorable performance-to-cost ratios, existing GPUs stockpiled by large AI organizations might return to the market.
Are blockchain-based GPU clusters solving a long-term problem? While blockchains can support any chip beyond GPUs, the actions of demand-side actors will ultimately determine the trajectory of projects in this space.
Conclusion
Fragmented networks with small GPU clusters won’t solve the problem. There’s no place for “long-tail” GPU clusters. GPU providers (retail or smaller cloud players) will gravitate toward larger networks offering better incentives. This will hinge on strong tokenomics and the ability of supply-side participants to support multiple compute types.
GPU clusters may face a consolidation fate similar to CDNs. If large players wish to compete with established leaders like AWS, they may begin sharing resources to reduce network latency and improve geographic proximity of nodes.
If demand grows larger (more models needing training, more parameters requiring computation), Web3 players must be extremely proactive on the supply-side business development front. If too many clusters compete for the same customer base, supply fragmentation will occur (rendering the whole concept ineffective), while demand (measured in TFLOPs) grows exponentially.
Io.net has already emerged from numerous competitors with an aggregator-first model. They’ve aggregated GPUs from Render Network and Filecoin miners for capacity while bootstrapping supply on their own platform. This could point to the winning direction for DePIN GPU clusters.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
IOSG Ventures
