Scaling the Blockchain for Tomorrow’s Internet

Let’s face it, blockchain technology simply isn’t living up to its promises right now. Claims of decentralizing the financial system and the Internet have been stopped in their tracks by one key problem: Scalability.

Distributed networks such as Bitcoin and Ethereum are well behind where they need to be, even for simple applications like payment networks. And you can forget about the global, real-time, AI-enabled applications soon to come as well. If decentralized networks are going to change the world and open up new economic opportunities, they need some serious improvements first.

That’s not to say, however, that there aren’t a few interesting ideas being developed out there to tackle this scalability problem. Some are seeking to improve existing blockchains, while others are leapfrogging blockchain technology altogether. Some are already looking ahead to build the infrastructure robust enough to allow for an AI-enabled future.

Blockchain for Payments

The simplest and most fundamental use case for blockchain technology is payments, and the big decentralized networks are struggling right out of the gates. Ethereum can manage around 10–15 transactions per second, orders of magnitude fewer than required by modern payment networks. Bitcoin is even further off the mark.

More Advanced Applications

If this isn’t bad enough, the promises of blockchain technology go way beyond just payments. Everything from decentralized social networks to video games are being developed to run on blockchain infrastructure.

Twitter handles tens of thousands of calls to its API every day. Those are things like tweets, updates, and searches, and many of calls would require blockchain transactions in a distributed application.

The current scale of payment and social networks is huge, but perhaps even more daunting for blockchain researchers is what’s to come. By 2020 there will be around 50 billion devices connected to the Internet. Most of these will be Internet of Things (IoT) devices. Things like smart bike locks, key rings, and kitchen appliances that are taking up a whole new life when connected online.

Most of these billions of devices are already communicating with each other, without human intervention. AI and machine learning are also becoming important to make this machine-to-machine communication more useful. The infrastructure needed to support these kinds of digital economies will have to be extremely fast.

Achieving this kind of scale in a secure, decentralized network is the greatest technical challenge that the crypto industry has faced thus far.

1. Scaling Ethereum

A good place to start is by expanding the capabilities of one of the most powerful blockchains in the world, Ethereum.

Currently on the Ethereum network, every node checks every transaction, change, and line of code that is executed on the blockchain. This is the requirement that makes Ethereum difficult to scale. Transactions on the Ethereum blockchain must propagate across the entire network before the next transaction can go though. This provides a lot of security, but it’s a huge scalability bottleneck.

Sharding

One idea to try and scale the Ethereum blockchain itself is called sharding. In computing, sharding means splitting up a database into smaller, more manageable parts. In terms of the blockchain, it means removing the requirement that every node process every transaction.

This would essentially split up the network into smaller sub-chains. Each sub-chain would have its own state and transaction history. Nodes only have to validate transactions that happen on their sub-chain, and can ignore the rest. Many shards of the network can operate semi-independently to increase transaction throughput.

Taking things off-blockchain

Another idea is to take some of the transactions and process them separately from the Ethereum blockchain. The Raiden Network is an idea that proposes to build separate “Raiden” nodes that operate alongside Ethereum nodes. They can process transactions with one another and periodically settle their state to the main blockchain. This drastically increases scalability as thousands, or even millions, of smaller transactions can be compressed into just one.

Plasma is a similar idea that suggests using smart contracts on the Ethereum mainchain to build child-chains. These child chains do their own processing, but settle their state periodically to the Ethereum mainchain. This means that large amounts of processing on a child-chain can effectively be compressed into a small number of main-chain transactions. Child-chains can even host their own child-chains in order to construct hierarchical structures of blockchains.

Challenges

Blockchain scalability cannot actually be “solved.” The blockchain trilemma states that a network can only have two of the following properties:

• Decentralization
• Security
• Scalability

In order to scale, Ethereum will have to trade off some level of decentralization or security. These are highly political topics and changes won’t come easily.

Another key challenge to scaling blockchain platforms like Ethereum and Bitcoin are the fact that they use the blockchain data structure. That is, a sequential chain of cryptographically linked blocks. The fact that transactions have to be added in blocks sequentially is a tight constraint. This is to avoid double spending, but it turns out that there are other solutions to this problem…

Beyond the Blockchain — DAG

The Internet of Things will require even more scalability. In a different approach to achieve this, some decentralized networks are abandoning blockchains all together.

IOTA is a crypto project that’s replacing the blockchain with a data structure called a Directed Acyclic Graph, or DAG. You can think of this like a generalization of the blockchain, where each transaction doesn’t necessarily have to link to the most recent transaction or last block. Instead of linking the blocks of transactions, individual transactions are linked to older transactions in a graph structure. Relaxing this requirement removes the block creation rate as a bottleneck, and means that transactions can be confirmed almost immediately.

There are also other efficiency gains from this approach. IOTA’s DAG model opens the door to zero transaction fees. With Ethereum and Bitcoin, you pay a fee to miners to validate your transaction. With IOTA, your transaction has to validate two other transactions in order to be included in the network. When you build an IOTA transaction you are actually doing the mining right then and there. Zero fees are important for the Internet of Things, as the “things” need to communicate with thousands of other “things” at a very high rate. Transaction fees make this uneconomical.

Challenges

At first glance, DAGs can seem like a natural improvement to blockchains, but they come with a hefty set of challenges. Many of which are still unsolved.

The first problem is mitigating spam. Transaction fees on the Ethereum network prevent spammers from clogging the network. A denial of service attack would simply be too expensive to be worth it. With free transactions, this isn’t the case. You can put through as many transactions as you can mine, maybe even enough to clog the network. Some people are arguing that this problem is more difficult to solve than the IOTA claims in their whitepaper.

Another challenge that has been pointed out by critics is keeping the nodes in a DAG network synchronized with each other’s centralization. Right now, IOTA uses a central coordinator to keep the DAG running efficiently.

Connecting Everything to Everything: Fetch

There are some projects that are striving for scalability even beyond what IOTA and DAG networks are hoping for.

Fetch is a project taking on the task of building the infrastructure for the new Internet. The plan is to go beyond just a ledger, and create an entire decentralized digital world. One where AI powered, autonomous “digital entities” can interact with each other. These entities can represent people, machines, services, assets, or data and use artificial intelligence technology to act on their own.

As CTO Toby Simpson put it, “A world that connects anything to anything and everything to everything.” As you can probably guess, a platform that connects everything to everything needs near-unlimited scalability. And that is exactly what the Fetch team claims to be close to achieving.

To achieve this, Fetch draws inspiration from both the blockchain and DAG data structures. The result looks like multiple blockchains running in parallel, with a mechanism to keep the chains synchronized. These parallel transaction chains are called “transaction lanes.” The more transaction lanes the Fetch network can maintain, the faster the network can go.

Figuring out which transaction should go to which lane is a difficult problem.

There is a transaction sequence:

1. Transactions are distributed among the resource groups,
2. Then their computing power is partitioned into resource lanes,
3. And each lane define how to operate transactions in its own way.

Transactions are separated, but their original order is saved. Thus, transactions need to be parallelized as much as possible while staying in a consistent order. To solve this, the team at Fetch have come up with their own take on the proof-of-stake algorithm.

Here, rather than solving hashing problems to mine new blocks, miners on the Fetch network compete to solve the transaction lane puzzle. The one that finds the optimal way to parallelize the transactions across the transaction lanes gets the reward.

This idea could be revolutionary. The Bitcoin mining network is so powerful that it will soon use 0.5% of the entire world’s electricity. Imagine if all that computing power could be used to actually make the network faster and more efficient.

Challenges

A useful proof-of-work algorithm has never been used before in this way. It will take a lot of experimenting and testing to make sure it behaves in the way the designers expect, especially if it were to come under attack.

With an idea like Fetch, the value comes from the possibility of interacting with millions of autonomous agents to find interesting new opportunities to trade and interact. The biggest challenge will be growing the network to a size where it’s actually useful.

Conclusion

To reach mainstream adoption, decentralized networks need to massively improve their scalability. So far, no one has cracked the problem, but a few projects promise to be close. On-blockchain and second layer technologies are being developed to allow Ethereum to handle the transaction throughput of modern Internet applications.

But even these solutions are a long way off in regards to the scalability that the emerging machine-to-machine Internet needs. New projects like IOTA and Fetch are developing new data structures and algorithms built specifically for this purpose. If their solutions are good enough, then we can look forward to some epic new Internet platforms. Ones that deliver improved communication and cooperation online, and ones that bring unimaginable economic possibilities to fruition.

About the author:

Kirill Shilov — Founder of Geekforge.io and Howtotoken.com. Interviewing the top 10,000 worldwide experts who reveal the biggest issues on the way to technological singularity. Join my #10kqachallenge: GeekForge Formula.