Paradigm: A Detailed Analysis of Ethereum's Historical Scaling Challenges and Their Solutions
TechFlow Selected TechFlow Selected
Paradigm: A Detailed Analysis of Ethereum's Historical Scaling Challenges and Their Solutions
EIP-4444 can address Ethereum's historical data growth issue and free up space for increasing the gas limit.
Navigating Web3 tides with focused insightsAuthors: Storm Slivkoff, Georgios Konstantopoulos
Translation: Luffy, Foresight News
History growth is currently the biggest bottleneck for Ethereum scalability. Surprisingly, history growth has become a larger issue than state growth. Within a few years, historical data will exceed the storage capacity of many Ethereum nodes.
The good news is:
-
History growth is an easier problem to solve than state growth.
-
Solutions are already under active development.
-
Solving history growth will alleviate the state growth problem.
In this article, we continue our investigation into Ethereum scalability from Part 1, shifting focus from state growth to history growth. Using granular datasets, our goals are 1) to technically understand Ethereum's scaling bottlenecks and 2) to help inform discussions around optimal Ethereum gas limits.
What is history growth?
History is the collection of all blocks and transactions executed across Ethereum’s entire lifecycle—the complete set of data from genesis to the current block. History growth refers to the accumulation of new blocks and transactions over time.
Figure 1 illustrates how history growth relates to various protocol metrics and hardware constraints on Ethereum nodes. Compared to state growth, history growth is constrained by a different set of hardware limitations. History growth stresses network I/O because new blocks and transactions must be propagated across the entire network. It also strains node storage space, as each Ethereum node stores a full copy of the history. If history grows fast enough to exceed these hardware limits, nodes will no longer be able to maintain stable consensus with their peers. For an overview of state growth and other scalability bottlenecks, see Part 1 of this series.
Figure 1: Ethereum scalability bottlenecks
Until recently, most of each node’s network throughput was used to transmit history (e.g., new blocks and transactions). This changed with the introduction of blobs in the Dencun hard fork. Blobs now account for a significant portion of node network activity. However, blobs are not considered part of history because 1) they are only stored by nodes for two weeks before being discarded, and 2) they do not require re-execution of data since genesis. Due to (1), blobs do not significantly increase the storage burden on individual Ethereum nodes. We will discuss blobs further later in this article.
In this article, we focus on history growth and examine its relationship with state. Since state and history growth share some overlapping hardware constraints, they are related issues—solving one can help address the other.
How fast is history growing?
Figure 2 shows the rate of history growth since Ethereum’s genesis. Each vertical line represents one month of growth. The y-axis shows the number of gigabytes added to history that month. Transactions are categorized by their “target address” and sized using RLP byte representation. Contracts that cannot be easily identified are labeled as “unknown.” The “other” category includes a range of smaller categories such as infrastructure and gaming.
Figure 2: Ethereum history growth rate over time
Key takeaways from the above chart:
-
History is growing 6–8 times faster than state: History peaked at 36.0 GiB/month recently and is currently at 19.3 GiB/month. State peaked at about 6.0 GiB/month and is currently at 2.5 GiB/month. A comparison between history and state growth and cumulative size will be discussed later in this article.
-
History growth was accelerating prior to Dencun: While state grew roughly linearly over the years (see Part 1), history exhibited super-linear growth. Given that linear growth in the rate leads to quadratic overall growth, super-linear growth in the rate results in more than quadratic total growth. This acceleration abruptly halted after Dencun. This marks the first time Ethereum has experienced a significant decline in history growth rate.
-
Rollups were responsible for most recent history growth: Each L2 posts copies of its transactions back to the mainnet. This generates large amounts of history, making rollups the largest contributor to history growth over the past year. However, Dencun allows L2s to publish transaction data via blobs instead of history, so rollups no longer generate most of Ethereum’s history. We’ll cover rollups in more detail later.
Who contributes most to Ethereum’s history growth?
The volume of history generated by different contract categories reveals how Ethereum usage patterns have evolved over time. Figure 3 shows the relative contributions of various contract categories. This is the same data as Figure 2, normalized.
Figure 3: Contributions to history growth by contract category
These data reveal four distinct eras of Ethereum usage patterns:
-
Early Era (purple): Ethereum’s first few years saw little on-chain activity. Most of these early contracts are now difficult to identify and are labeled as “unknown” in the chart.
-
ERC-20 Era (green): The ERC-20 standard was finalized at the end of 2015 but didn’t gain traction until 2017–2018. ERC-20 contracts became the largest source of history growth by 2019.
-
DEX / DeFi Era (brown): DEX and DeFi contracts appeared on-chain as early as 2016 and started gaining attention in 2017. But it wasn’t until the DeFi Summer of 2020 that they became the top category for history growth. DeFi and DEX contracts accounted for over 50% of history growth during parts of 2021 and 2022.
-
Rollup Era (gray): In early 2023, L2 rollups began executing more transactions than the mainnet. In the months leading up to Dencun, they generated about two-thirds of Ethereum’s history.
Each era represents increasingly complex usage patterns on Ethereum. Over time, this increasing complexity can be seen as a form of scaling that isn't captured by simple metrics like transactions per second.
In the most recent data month (April 2024), rollups no longer generate the majority of history. It remains unclear whether future history will come primarily from DEX and DeFi or if new usage patterns will emerge.
What about blobs?
The Dencun hard fork introduced blobs, significantly changing the dynamics of history growth by allowing rollups to publish data cheaply via blobs instead of history. Figure 4 zooms in on the history growth rate before and after the Dencun upgrade. This chart is similar to Figure 2, except each vertical line now represents one day rather than one month.
Figure 4: Impact of Dencun on history growth
From this chart, we draw several key conclusions:
-
Since Dencun, rollup-related history growth has dropped by about two-thirds: Most rollups have transitioned from calldata to blobs, drastically reducing the amount of history they generate. However, as of April 2024, some rollups still haven’t switched from calldata to blobs.
-
Total history growth has decreased by about one-third since Dencun: Dencun only reduced rollup-related history growth. History growth from other contract categories slightly increased. Even post-Dencun, history growth remains about 8 times faster than state growth (more details in the next section).
Although blobs have slowed history growth, they remain a new feature on Ethereum. It remains unclear at what level history growth will stabilize in the presence of blobs.
How fast is acceptable for history growth?
Raising the gas limit increases the history growth rate. Therefore, proposals to raise the gas limit (such as Pump the Gas) must consider the relationship between history growth and per-node hardware bottlenecks.
To determine an acceptable history growth rate, we must first understand how long current node hardware can sustain existing network and storage demands. Networking hardware may indefinitely support the status quo, as history growth is unlikely to return to pre-Dencun peaks before any gas limit increase. However, the storage burden of history accumulates over time. Under current storage practices, each node’s hard drive will inevitably fill up with history.
Figure 5 shows the storage burden on Ethereum nodes over time and projects growth over the next three years. The projection uses the April 2024 growth rate. This rate may rise or fall depending on future usage patterns or gas limit changes.
Figure 5: Size of history, state, and full node storage burden
From this figure, we derive several key conclusions:
-
History takes up about three times as much storage as state. This gap will widen over time, as history grows about eight times faster than state.
-
1.8 TiB is a critical threshold—many nodes will be forced to upgrade their storage drives. 2TB is a common hard drive size, offering only about 1.8 TiB of usable space. Note that TB (trillion bytes) and TiB (= 1024^4 bytes) are different units. For many node operators, the “true” threshold is even lower, as post-merge validators must run both execution and consensus clients together.
-
This threshold will be reached within 2–3 years. Any increase in the gas limit will accelerate its arrival. Reaching this point will impose non-trivial maintenance burdens on node operators and require additional hardware purchases (e.g., a $300 NVMe drive).
Unlike state data, history data is append-only and accessed far less frequently. Thus, in theory, history can be stored separately from state on cheaper media. Some clients, such as Geth, already support this.
Beyond storage capacity, network I/O is another major constraint on history growth. Unlike storage, network I/O does not pose immediate problems for nodes in the short term, but it will become important when considering future gas limit increases.
To understand how much history growth typical Ethereum nodes’ network capacity can support, we need to know the relationship between history growth and various network health metrics—such as reorg rates, slot misses, attestation misses, sync committee misses, and block submission latency. Analyzing these metrics is beyond the scope of this article but can be explored in prior research on consensus layer health. Additionally, the Ethereum Foundation’s Xatu project continues building public datasets to accelerate such analysis.
How can we solve history growth?
History growth is an easier problem to solve than state growth. It could be nearly fully addressed by the proposed EIP-4444. This EIP changes each node’s requirement from storing the entire Ethereum history to storing only one year of history. After implementing EIP-4444, data storage would no longer be a scalability bottleneck, and gas limit increases would no longer be constrained in the long term. EIP-4444 is necessary for the network’s long-term sustainability; otherwise, history growth would force regular node hardware upgrades.
Figure 6 shows the impact of EIP-4444 on per-node storage burden over the next three years. This chart is identical to Figure 5 but includes lighter lines representing storage burden after EIP-4444 implementation.
Figure 6: Impact of EIP-4444 on Ethereum node storage burden
Key conclusions from this chart:
-
EIP-4444 will halve current storage burden. It reduces storage from 1.2 TiB to 633 GiB.
-
EIP-4444 will stabilize history storage burden. Assuming constant history growth, data will be pruned at the same rate it is generated.
-
After EIP-4444, it will take many years for node storage burden to reach today’s levels. This is because state growth will be the only factor increasing storage burden, and state grows much slower than history.
Even after EIP-4444, nodes will still store one year of history, which imposes some storage cost. However, even at global scale, this burden will be manageable. Once the historical data preservation method proves reliable, the one-year expiry in EIP-4444 could potentially be shortened to months, weeks, or even less.
How should Ethereum’s history be preserved?
EIP-4444 raises a question: if nodes themselves no longer preserve history, how should it be preserved? History plays a central role in Ethereum’s validation, accounting, and analysis, so preserving it remains crucial. Fortunately, history preservation is a simpler problem—it only requires 1/n honest data providers. This contrasts sharply with state consensus, which requires 1/3 to 2/3 honest participants. Node operators can verify the authenticity of a historical dataset by 1) replaying all transactions since genesis and 2) checking whether those transactions reproduce the same state root as the current chain tip.
There are multiple ways to preserve history:
-
Torrents/P2P: Torrents are the simplest and most reliable method. Ethereum nodes could periodically package portions of history and share them as public torrent files. For example, a node might create a new history torrent file every 100,000 blocks. Clients like Erigon already perform this process to some extent in a non-standardized way. To standardize it, all node clients must use the same data format, parameters, and P2P network. Nodes could choose whether to participate based on their storage and bandwidth capacity. The advantage of torrents lies in leveraging a highly robust open standard already supported by extensive tooling.
-
Portal Network: The Portal Network is a new network designed specifically to host Ethereum data. Similar to torrents, it offers additional features that make data verification easier. The Portal Network’s advantage is that these extra verification layers provide utility for light clients, enabling efficient validation and querying of shared datasets.
-
Cloud Hosting: Cloud storage services like AWS S3 or Cloudflare R2 offer low-cost, high-performance options for preserving history. However, this approach introduces greater legal and operational risks, as there’s no guarantee these providers will always be willing or able to host cryptocurrency data.
The remaining implementation challenges are more social than technical. The Ethereum community needs to coordinate on specific implementation details to integrate them directly into every node client. In particular, performing a full sync from genesis (instead of a snapshot sync) will require retrieving history from historical data providers rather than Ethereum nodes. These changes do not technically require a hard fork and could therefore be implemented earlier than Ethereum’s next hard fork, Pectra.
All these history preservation methods could also be used by L2s to store the blob data they publish to the mainnet. Compared to history preservation, blob preservation is 1) harder due to the much larger total data volume, and 2) less critical, as blobs are not required to replay mainnet history. However, blob preservation remains necessary for each L2 to replay its own history. Therefore, some form of blob preservation is important for the entire Ethereum ecosystem. Moreover, if L2s develop robust blob storage infrastructure, they may also be able to easily store L1 history data.
It would be helpful to directly compare the datasets stored by various node configurations before and after EIP-4444. Figure 7 shows the storage burden across different types of Ethereum nodes. State data includes accounts and contracts, history data includes blocks and transactions, and archive data consists of optional data indexes. The byte counts in this table are based on a recent reth snapshot, though numbers for other clients should be roughly similar.
Figure 7: Storage burden across different Ethereum node types
In other words:
-
Archive nodes store state, history, and archive data. Archive nodes are used when someone needs to easily query historical chain states.
-
Full nodes store only state and history data. Most nodes today are full nodes. Full nodes have about half the storage burden of archive nodes.
-
Post-EIP-4444 full nodes store only state and the most recent year of history. This reduces node storage burden from 1.2 TiB to 633 GiB and stabilizes history storage at a steady-state value.
-
Stateless nodes, also known as "light nodes," store no datasets and can validate instantly at the chain tip. This node type becomes possible once Verkle tries or other state commitment schemes are added to Ethereum.
Finally, there are additional EIPs that could limit the history growth rate, rather than just adapting to it. These would help keep network I/O within bounds in the short term and storage within limits in the long term. While EIP-4444 remains essential for long-term network sustainability, these other EIPs will help Ethereum scale more efficiently in the future:
-
EIP-7623: Recalibrates calldata pricing to make certain calldata-heavy transactions more expensive. Making these usage patterns costlier will push some to shift from calldata to blobs, thereby reducing history growth.
-
EIP-4488: Imposes a limit on the total calldata allowed per block. This places stricter caps on the rate of history growth.
These EIPs are easier to implement than EIP-4444 and could serve as short-term stopgaps before EIP-4444 goes live.
Conclusion
The purpose of this article is to use data to understand 1) how history growth works and 2) how to solve it. Much of the data presented here is difficult to obtain through conventional means, so we hope publishing it provides fresh insights into the history growth challenge.
History growth as a scalability bottleneck has not received sufficient attention. Even without increasing the gas limit, Ethereum’s current practice of storing full history will force many nodes to upgrade hardware within a few years. Fortunately, this is not a hard problem to solve. A clear solution already exists in EIP-4444. We believe this EIP should be accelerated to make room for future gas limit increases.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
Paradigm
