Understanding Walrus, Sui's New Decentralized Storage Solution
TechFlow Selected TechFlow Selected
Understanding Walrus, Sui's New Decentralized Storage Solution
Sui development team product, independent PoS chain, new governance token WAL, potential airdrop opportunity.
Navigating Web3 tides with focused insightsBy Alex Liu, Foresight News
The decentralized storage network Arweave launched its compute layer AO, successfully driving a resurgence in AR's price, ecosystem activity, and overall attention—a true comeback. Now, as a general-purpose computing chain, what impact will Sui create by launching its own decentralized storage network, Walrus?
Background
Team
The company behind Solana is called Solana Labs; Aptos Labs backs Aptos; and the team behind Sui is Mysten Labs—distinctive as ever. Most of Mysten Labs’ founders and employees come from Diem, the blockchain project formerly developed by Facebook (now Meta), which was later disbanded.
Walrus is the latest product categorized by Mysten Labs under "protocols and platforms"—a decentralized storage network. The name "Walrus" (meaning “walrus” in English) carries slogans on its official website such as “robust like a walrus” and “adaptable like a walrus,” conveying the protocol’s reliability and usability as a storage system.
Relationship with Sui
Walrus is built on top of Sui and leverages Sui to coordinate the sale of storage space and metadata. However, using Walrus does not require building applications or products on Sui. Additionally, the new governance token WAL will serve as a utility token, not SUI.
Competitive Landscape
Decentralized storage protocols typically fall into two main categories. The first category consists of fully replicated systems, with Filecoin and Arweave being prime examples in this space. A key advantage of this approach is full file availability across storage nodes, enabling easy access and migration even if individual nodes go offline. This setup supports permissionless participation since nodes don’t need to rely on each other to recover files.
The reliability of these systems depends on the robustness of selected storage nodes. Under the classical static adversary model assuming one-third malicious actors and an infinite pool of candidate storage nodes, achieving "twelve nines" security (i.e., less than a 10^-12 probability of losing file access) requires storing more than 25 copies across the network—resulting in a 25x storage overhead. Moreover, such systems are vulnerable to Sybil attacks, where bad actors may falsely claim to store multiple copies of a file, undermining system integrity.
The second category uses Reed-Solomon (RS) coding for decentralized storage services. RS coding divides files into smaller segments known as shards, each representing part of the original file. As long as the total size of available shards exceeds the original file size, the original can be decoded. However, RS coding has drawbacks. Encoding and decoding involve field operations, polynomial evaluation, and interpolation—computationally expensive processes. These operations are only practical when the field size and number of shards are relatively small, limiting both the maximum file size and the number of participating storage nodes. Otherwise, encoding costs become prohibitively high, restricting decentralization. Another issue arises when a storage node goes offline and needs replacement: unlike fully replicated systems, data cannot simply be copied from one node to another. In RS-coded systems, all existing storage nodes must send their shards to the replacement node so it can reconstruct the missing shard. This recovery process results in O(|blob|) data transmission over the network. Frequent recovery operations undermine the storage savings achieved by reducing replication.
Challenges in Storage
Beyond replication methods, all current decentralized storage systems face two additional challenges:
-
They require continuous challenges to verify that storage nodes retain data without dropping it. This is crucial in open, decentralized systems where payment is provided for storage, but currently limits scalability because each file requires individual verification challenges.
-
Storage nodes need coordination: knowing who is in the system, which files have paid for storage, implementing incentive mechanisms, managing challenges, and mitigating abuse. This is why each of the above systems implements a custom blockchain to execute transactions and introduces a cryptocurrency separate from the storage protocol itself.
Core Innovations
Facing these challenges, how does Walrus innovate to offer a different solution for decentralized storage?
In short:
By adopting innovative erasure coding techniques, Walrus rapidly and reliably encodes unstructured data blocks into smaller shards distributed across a network of storage nodes. Even if up to two-thirds of the shards are lost, the original data block can still be quickly reconstructed using partial shards. This is achievable while maintaining a replication factor of only 4x to 5x—comparable to existing cloud services—while offering the benefits of decentralization and broader fault tolerance.
More specifically:
Walrus introduces RedStuff, a novel 2D encoding algorithm designed for Byzantine Fault Tolerance (BFT). Based on fountain codes, RedStuff combines fast operations with high reliability.
RedStuff uses simple operations—primarily XOR (exclusive OR)—to encode data into primary and secondary shards. These shards are distributed across storage nodes, with each node holding a unique combination. For different dimensions of encoding, RedStuff employs different thresholds. The primary dimension uses an f+1 recovery threshold, allowing asynchronous writes since only 2f+1 signatures are needed to prove data block availability—already establishing a 3x replication factor.
The secondary dimension uses a 2f+1 recovery threshold. This design achieves asynchronous proof-of-storage for the first time while adding only a 1.5x overhead, resulting in a total replication factor of less than 5x. More importantly, lost shards can be recovered based on the amount of lost data, saving bandwidth—thanks to 2D encoding.
RedStuff advantages include: compared to RS coding, the use of simple XOR operations enables faster encoding/decoding; low storage overhead allows the system to scale to hundreds of nodes, providing high elasticity and fault tolerance, ensuring data recovery even under Byzantine failures.
As a permissionless protocol, Walrus includes an efficient committee reconfiguration protocol to handle natural node churn and ensure continuous data availability. When a new committee replaces the current one between epochs, the reconfiguration protocol ensures all data blocks past their point of availability (PoA) remain accessible. RedStuff’s 2D encoding makes state migration more efficient—even if some nodes are unavailable, others can recover lost shards.
Node 1 and Node 3 help Node 4 recover shard data
Walrus introduces an asynchronous challenge protocol to verify whether nodes are correctly storing data. This protocol enables efficient proofs of storage, ensuring data availability without relying on network assumptions, with costs scaling logarithmically with the number of stored files.
Walrus’ economic model is based on staking, incorporating both rewards and penalties. An innovative storage attestation mechanism scales logarithmically with the number of stored files, reducing the cost of proving file storage.
In summary, Walrus centers around the RedStuff protocol, delivering a scalable, resilient, and economically viable decentralized storage solution capable of providing high authenticity, integrity, auditability, and availability at reasonable cost.
All of this is made possible by Sui serving as Walrus’ control layer. Having a scalable, programmable, and secure infrastructure as a coordination layer allows Walrus to focus on solving core problems in decentralized storage.
Potential Airdrop
Walrus will launch its independent token WAL, with utilities including staking and governance. How can one qualify for a WAL airdrop? By referencing AO’s distribution method, holding SUI may be one potential way.
Walrus is expected to release its testnet soon, with mainnet launch timing yet to be announced. For now, users can refer to the official documentation to learn how to deploy their own website using Walrus.
Sources:
Walrus Whitepaper:
https://docs.walrus.site/walrus.pdf
Walrus: Decentralized Storage and DA Protocol, Can Build L2s and Large-Scale Storage on Sui: https://foresightnews.pro/article/detail/63040
Mysten Labs Researcher X Thread:
https://x.com/LefKok/status/1836868240666153293
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
Foresight News
