As some of you probably already know, we recently celebrated the tenth anniversary of Bitcoin. During the time that has passed since Nakamoto’s historic white paper publication, the number of blockchain-based projects has grown to the thousands. All of them, much like their ancestor, face the challenge of creating a secure, fast, and scalable decentralized system. This is, without a doubt, a challenging task and for blockchain-architecture systems today, it is far from being solved. The problem is that all such networks inevitably face the “blockchain trilemma”, a phrased was first formulated by Vitalik Buterin. That’s why engineers today are developing alternative solutions in order to reach consensus faster on the network.
https://www.jeffersoncapital.info/the-scalability-trilemma/
This article will give an overview of how network sharding is able to secure explosive scalability and how these technological advances can be implemented in decentralized, content-oriented platforms like BOLT.
“Software is simply the encoding of human thought, and as such has an almost unbounded design space.”
Chris Dixon
Fighting the blockchain trilemma
It’s easy to create a centralized system where all participants share one “right” view on reality, i.e. achieve consensus. However, this view will be dictated from some single center that governs the network, thus the nodes have to trust it. In a decentralized system, however, reaching consensus is a much more tricky task.
The first and main problem that was solved by the creator (or possibly creators) of Bitcoin is not directly related to its function as a digital currency. This problem is fundamental and arises in many areas of social life: how to organize interaction in the system of elements, assuming that each of them is potentially unreliable, so that the system comes to a single solution? Speaking even more generally, we need to invent a way to ensure the whole system attack and failure tolerance, taking into account the fact that its elements may be egoistic or unreliable.
To imagine a secure, scalable, and fast decentralized system of interacting agents from scratch is not an easy task. So let us first take a look at the well-known technological postulates underlying the architecture of Bitcoin.
- First of all, all transactions on the network are combined into blocks and processed together to save time.
- These blocks are time-stamped and linked in a series (or a chain) via cryptographically protected hash-functions, so that no one can change the history recorded in the blockchain.
- In order to ensure the passage of transactions on the network, miners perform the massive computations needed to calculate new block hashes.
- A copy of the distributed registry is stored on each full node of the network and to rewrite the history the attacher would have to take control of over more than half of the network power.
- Such a “51% attack” is extremely difficult to perform because of Bitcoin’s global prevalence and the associated high network hashrate.
https://www.blockchain.com/en/pools
As a result, the network is controlled by open-source Bitcoin code only, which is absolutely transparent for everyone, bringing decentralization to the world of centralized institutions and third-party intermediates.
Sounds good, right? Well, by following these principles we inevitably fall into the Bermuda Triangle of the blockchain trilemma. It states that a distributed system based on the blockchain can have a maximum of two of the three most important properties: scalability, security, and decentralization. For example, Bitcoin being a secure system with quite good decentralization suffers from the slowness of transactions — because all of the full nodes on the network must come to a consensus. Recall that in December 2017, when the network was significantly overloaded, transactions were pending for many hours and the fee reached $40.
These problems are fundamental and are an inevitable part of blockchain architecture. In the recent years, several scientists and software engineers have claimed that the architecture described above is not the only one possible.
Look at the birds in a large flock, creating fascinating 3D figures in the sky. How do they agree with each other to act in a coordinated way? The answer is: each bird remembers the location of several of its nearest neighbors in the flock and moves with them, contributing to the overall whole. A single bird does not broadcast where it will fly to everyone in the flock, and in such a distributed system there is no consensus in the usual meaning of the word. However, a flock demonstrates perfect synchronization that reacts to external stimuli and dangers (i.e. predators) rapidly and effectively.
What if we started creating our “artificial” distributed systems based on natural principles? Let’s ask ourselves: does each network node really need to communicate with all the others and also keep the full history of the network? We may, for example, divide the network into several smaller ones, so that each node communicates mainly with the participants of its “own” subnet, just like a separate bird in a large flock. The information flow between different subnets will be much less intensive than in a single networks, and this will open the road to scalability.
https://www.oreilly.com/library/view/algorithms-and-parallel/9780470934630/c07-sec1-0008.xhtml
There is one strong idea that is set to potentially overcome the blockchain trilemma — sharding.
Sharding: divide & conquer
The concept of sharding as the idea of dividing a large amount of data into parts is not new. It has been successfully used in “traditional” centralized systems as a method of horizontal partitioning of data within a database. Despite the fact that database sharding is not a simple matter, in general, it is relatively easy to divide the data into several parts if the system has a single control center.
Another thing is to implement sharding in a decentralized network in which nodes can appear and disappear continuously and the load on the blockchain is constantly changing.
When Vitalik Buterin described the future of sharding in the Ethereum ecosystem, he said, “Imagine that Ethereum has been split into thousands of islands. Each island can do its own thing. Each of the islands has its own unique features and everyone belonging on that island i.e., the accounts, can interact with each other AND they can freely indulge in all its features. If they want to contact other islands, they will have to use some sort of protocol.”
Sounds like a simple and easy way to increase network scalability, doesn’t it? Unfortunately, it’s not that simple. While each full node on the network is forced to store a full record of the blockchain of its distributed system, scalability will be limited and its bottleneck will be the contacts between the different shards.
Imagine an apartment building where all residents have the strange task of maintaining strict accounting records, tracking all transactions made amongst each other. In this analogy:
- The people in the apartments are separate shards of the network, i.e. close communities that communicate with the rest of the inhabitants.
- Lending and borrowing money stands for performing transactions.
- The house and its inhabitants is the whole community using the network.
Imagine that each resident is forced to keep a full log of all transactions, walking through other people’s apartments and asking who and how much was borrowed today. Additionally, every hour all residents are required to gather together and check their registers, coming to a consensus on the history of the network and writing it to a single “main” register (the blockchain).
Is there any sense in such kind of “sharding”? Will such a decentralized system be effective? Of course not. In order for the scalability effect of sharding to be fully manifested, it is necessary to radically reduce the number of contacts of nodes from the different shards. For example, to allow each family to record transactions only inside their apartment and to provide the accumulated information to the rest at a monthly meeting.
This direction is taken, for example, by the Zilliqa project. The key feature is that although each node is still cognizant about the current network state, the full history of transactions doesn’t need to be stored. Instead, it’s split into pieces and stored separately throughout different parts of the network.
This achievement became possible due to Zilliqa’s special consesus algorithm that is based on the Bysantine Fault Tolerance. As the team states, “We choose BFT to design our consensus protocol to ensure that the resulting blocks are definitive, without the need of long confirmation times as required in the popular ‘longest-chain’ rule in existing cryptocurrencies.” The main feature of this consensus protocol is its finality: once the block is broadcasted to the blockchain, no other block can claim to have the same parent block. So there is no need for confirmations and for any Proof of Work to achieve agreement on the transaction (speaking figurally, PoW is a mechanism designed to prune wrong chains, which are absent here).
Because of the finality feature, the entire transaction history does not have to be saved every time, it is instead sufficient enough to store only the latest state of the network in each node. This direction seems to be very promising and can really help solve the urgent problem of scalability.
If this approach is successful, distributed systems based on sharding can be widely used in areas where a large number of microtransactions are required, such as content-oriented platforms.
Handling a decentralized media content system requires incredible transaction speeds and it also imposes strict requirements for network scalability. Indeed, if the content-sharing platform is successful, the number of its users will grow, and what worked for a 1,000-person network will likely not work for one with a million nodes. That’s why BOLT decided to build their ecosystem on Zilliqa — enabling a decentralized mobile-first platform to enhance the digital entertainment experience for their three million plus existing users in the emerging markets across East Africa and Southeast Asia.
BOLT’s mission is to give underbanked/unbanked users access to truthful, curated digital content and entertainment. Its network will be designed to bridge the gaps that exist amongst digital services and their potential users in developing markets. As the BOLT white paper says,
“… users can choose how the platform operates for them — they can be contributors or consumers, but the super-profits currently earned by centralised social media platforms will not exist.”
The BOLT token services allow content producers to be able to monetize their efforts without the need for distributors and third-party intermediaries. This goal requires a huge number of micro-transactions, therefore the BOLT network was an ideal candidate to be powered by Zilliqa’s blockchain sharding solution. Because Zilliqa has successfully implemented sharding on its network, transactions in the BOLT ecosystem will be lightning fast and the BFT consensus algorithm will make them safe for both users and content creators.
All existing blockchain architectures suffer from the blockchain trilemma, which limits their most important properties. One of the most serious attempts to get around these problems today is sharding and applications such as content-oriented platforms are in dire need of its benefits. There are already platforms that have effectively implemented this technology in reality, achieving an exponential increase in network bandwidth. We’ll see much more in the future when such systems can be used to build ecosystems with a large number of small transactions. In this case, decentralization of the user-facing applications is much closer than we think!
About the author:
Kirill Shilov — Founder of Geekforge.io and Howtotoken.com. Interviewing the top 10,000 worldwide experts who reveal the biggest issues on the way to technological singularity. Join my #10kqachallenge: GeekForge Formula.