Welcome to the Age of the Decentralized Storage Systems

From Protocol Layers to Application Layers Storage Systems

Introduction

Recently, blockchain technology has become the home to multiple distributed and decentralized file storage systems architectures -- either as smart contracts or a distributed file network system.

The decentralized and distributed file storage system, like others, has its strengths and weakness.

The decentralized file storage system provides extensive flexibility for users and the ability to participate; however, some of the decentralized file storage systems nodes are overly reliant on a centralized architecture.

Decentralization is a complex socio-technical concept that is interpreted based on its functionality, from protocol layers to application layers.

Conceptually, decentralization is an old concept re-enacted in an emerging blockchain environment, which simply means, the transfer of authority from a centralized entity to a distributed, peer-to-peer, and trustless network of computer nodes operating across the world without central databases.

The paradigm-altering mechanism of decentralization reduces consensus blockchain protocols’ point of failure.

• Protocol layers: Referred to collectively architectural stacks that define the foundational structure and design of a network. For example, the foundation of the point-to-point internet is referred to as a layer 0 protocol while, programmable blockchains protocols such as Ethereum and other base layer protocols are referred to as layer 1.

  
  

• Application layers: Are the applications built on programmable protocols such as smart contracts and storage systems built on layer 1 blockchain or the likes of Facebook, Google, and Twitter built on a layer 0 internet that operates using protocols such as SMTP, TCP/IP, and HTTP.

  
  

• Federated systems: A cohesive and homogenous unit of smaller subunits of a system that collaborate to form a holistic, but retain significant local autonomy.

The fundamental and architectural principles of decentralization are built on consensus mechanisms deployed by programmable blockchain protocols, which remove intermediaries and central entities from controlling the network.

However, there have been several misconceptions when discussing decentralization, as most people can’t distinguish the differences between decentralized consensus protocols and the decentralized applications (DAPP) built on these protocols.

Furthermore, some of the applications built on decentralized consensus protocols are partly centralized by design, while most are fully decentralized.

Thus, understanding the fundamental principles of consensus blockchain protocols and the functionality of decentralized applications built on these protocols is essential.

Furthermore, exploring other decentralized entities such as “quicknode” and “thegraph” that are providing additional service that enables both users and organizations to power their blockchain, nodes, and use API to query indexed pending on-chain transaction.

Hence, this article will help in providing a clear understanding by explaining the common misconceptions surrounding centralization, decentralization, web3, and blockchain technology. However, some blockchain protocol websites, architectures, and nodes are still running on centralized Amazon Web Services (AWS) servers.

Peer-2-Peer Decentralized File Storage

Decentralized file systems have become prevalent since the mass adoption of blockchain technology; unlike the web2.0 distributed file-sharing systems like Bit-torrent and Napster, blockchain file storage systems require full decentralization with no point of failure.

Protocol and Application Layer (Thin Protocols and Fat Applications)

Applications Layer (Fat Application Thin Protocols)

In the early evolution of the shared internet protocol of web1.0 and web2.0, there was a clear distinction between the application layers like Facebook, Twitter, Amazon, and Google in terms of success distribution from the protocols layers such as TCP/IP, HTTP, SMTP. This phenomenon is referred to by Joel Monegro (2016; 2020) as “thin” protocols and “fat” applications.

Joel Monegro stated that the protocol layer on web1.0 and web2.0

“produced an immeasurable amount of value. But most of it got captured and re-aggregated on top at the applications layer, largely in the form of data (think Google and Facebook)”

The majority of the early web1 and web2 users doesn’t understand how layer zero point to point internet relay information through Hypertext Transfer Protocol (HTTP) server, how Domain Name System (DNS) resolves IP addresses to a readable domain name, and how Simple Mail Transfer Protocol (SMTP) function to relay email from one server to another. Yet they enjoyed the seamless service provided.

The protocol layer of web1.0 and web2.0 was decentralized to an extent, such as DNS.

Protocols Layer (Fat Protocols Thin Application)

Consequently, as the adoption of consensus blockchain technology, becomes prevalent, the protocol becomes “FAT” while the applications built on the protocols are considered “Thin”.

Protocols such as bitcoin, Ethereum, Polkadot, and other has enjoyed early dominance in terms of market capitalization compared to the application built on them.

Fundamentally, the reversed world of consensus blockchain protocols where the value proposition is concentrated at the shared protocol level, while only a small portion of the value is allocated to the applications (Thin Applications).

Types of Blockchain

• Public blockchain: Decentralized, permissionless (openly accessible for anybody to interact with without approval, censorship resistance, running on a peer-to-peer node, smart contracts) No point of failure.

  
  

• Private blockchain: Centralized, permissioned, interaction is private and requires permission to participate with a point of failure.

  
  

• Consortium blockchain: semi-permissioned.

  
  

Decentralized Storage Solution

Decentralized storage systems are network or blockchain-based peer-to-peer networks of user-operators (nodes) who control the distributed storage of data in different locations.

This leads to the creation of a resilient distributed file storage sharing system where the data and the file are stored all over the world.

Decentralized file storage system running peer-2-peer machines (nodes) operating and storing data from various locations working simultaneously.

The Blockchain-Based File Storage System

The Arweave Network is now running Succinct Random Proofs of Access (SPoRA). Succinct Proofs of Random Access (SPoRA) is a new type of consensus mechanism for the Arweave network.

Before migrating to the Succinct Random Proofs of Access, Arweave was running a classic Proof of Work (PoW) consensus mechanism, with the additional requirement for blocks to include a reference to past data on the network known together as Proof of Access (PoA).

• Arweave: This is a decentralized file storage system that enables you to store documents and applications forever.

  
  

• Ethereum: Decentralized application file storage system with a design mechanism to host a smart contract.

  
  

Distributed Network File Storage System

• Inter-planetary file system (IPFS): IPFS is a decentralized web file storage protocol, aiming to serve as a web2.0 HTTP protocol in the web3 environment that points users to their data through hash to locate the nodes storing the content without any incentives.

  
  

  This leads to the development of additional IPFS layers such as filecoin to incentives participants. IPFS uses content addressing storage patterns to decentrally store cryptographic compression content into a long string of data by referencing a file by its unique content.

  
  

• Pinata

  
  

The Contract-Based Decentralized File Storage System

• Filecoin: offers decentralized storage and cloud services globally. With the largest market share in terms of storage consumption and network capacity. The filecoin architecture is built on top of the Interplanetary File System (IPFS), which serves as the network’s distributed data storage and sharing layer for filecoin.

  
  

  In its attempt to become the prominent web3 cloud storage and decentralized alternative to centralized web2.0 platforms, filecoin is considering 4 additional use cases categories (1) NFT and Web3 storage; (2) permanent storage and Web2 datasets. (3) metaverse and gaming; and (4) audio and video.

  
  

• Skynet

  
  

• Siacoin

  
  

• Storj: Storj network uses object stores to cryptographically encrypt, shard, and distribute stored data to peer-2-peer nodes. Storj is aiming to replace one of the widely used centralized cloud and storage solutions, Amazon S3, hence, uses S3-compatible APIs.

Conclusion

Decentralization goes beyond the democratization of participation or distribution of a peer-2-peer network of nodes.

In consensus blockchain protocols, the decentralized storage system is an important issue since blockchain has limited on-chain storage capability; hence, the decentralized and distributed file storage system is important for data storage.

The majority of NFTs and on-chain files are stored with IPFS and Arweave, to name a few because storing files directly on the blockchain could be very expensive and result in data latency. Hence, there is a need for a decentralized and distributed file storage system to enhance the storage system in the ecosystem.

The article was also published here.