How We're Addressing Blockchain's Low TPS Problem While Retaining Its Core Values [Part I]

As we know, blockchain is now suffering from its low transactions per second (TPS). For example, Bitcoin only processes 6 to 7 TPS, and Ethereum currently processes roughly 15 TPS, while Alipay — a centralized payment system — was able to deliver 200,000+ TPS on November 11, 2017.

The low TPS of blockchain greatly limits its wider applications.

Even though the centralized systems could offer great TPS nowadays, scalability has been a core problem in the centralized world due to the huge population of Internet users.

To meet the needs of all Internet users, a large number of computer scientists and engineers from Internet companies such as Google and Facebook, open source community, and academia have spent enormous efforts and resources on scalability, producing various highly-scalable systems and verifying them in production for years.

Some notable projects include Google File System, BigTable, Hadoop, Cassandra, etc.

QuarkChain aims to address the blockchain scalability problem by employing above mentioned technologies. However, among these centralized scalability technologies, many of them are not best fit for the decentralized world and cannot be applied directly.

To scale blockchain while maintaining the core values of blockchain — decentralization and security, we have to carefully select the technologies that are best fit to blockchain, integrate them together, and apply them to the decentralized world.

Following this way also means that we have to build the infrastructure from scratch as there is no example in the decentralized world. But the good thing is that our team members are senior software engineers from Google, Facebook, and Uber, who have extensive experience in designing, testing, maintaining, and optimizing their scalable systems.

The unique experience and background allow us to develop QuarkChain in a very fast pace and to avoid a lot of pitfalls that are discovered in centralized systems in the early stage of QuarkChain development.

To future discuss scalability technologies behind QuarkChain in details, let us first briefly review the history of scalability in the centralized world.

Brief History of Scalability in the Centralized World

Before the era of the Internet (2000s), in order to satisfy the performance needs of growing services, people mostly rely on a general class of scalability called vertical scalability (or scale up). The basic idea of vertical scalability is to improve a single node system by putting more advanced/expensive hardware:

• Add faster CPUs for computation bottleneck; or
• Add more memory for memory/cache bottleneck; or
• Add more or faster disks for storage bottleneck.

There are a couple of companies that are experts in vertical scalability in different areas such as:

• Oracle, which is famous for its database;
• SUN, which is famous for its workstation;
• EMC, which is famous for its storage.

The main benefit of vertical scalability is that it is relatively simple to manage — merely adding more resources. However, there are a couple of major disadvantages:

The upfront cost of vertical solutions can be high. For example, an entry level of EMC storage can be much more expensive than a commodity PC.Adding more capacity in vertical scalability can be costly when the capacity approaches the ceiling.

Due to physical limits of hardware resources, increasing 2x capacity of a vertical system costs much more than 2x resources most of time.

For example, from ark.intel.com on June 11, 2018, an Intel Xeon E5–1620 v4 with 4 cores and up to 3.80 GHz per core is about $300 (total 15.2GHz with $19.7 per GHz) while Intel Xeon E5–1680 v4 with 8 cores and up to 4.00 GHz per core is about $1723 (total 32GHz with $53 per GHz).

With the explosion of the applications of Internet, the number of users grows exponentially over time, and serving all the users via vertical solutions becomes extremely costly and sometimes even impossible.

To meet the service needs of global-wise users, new Internet companies such as Google and Facebook resort to horizontal scalability (scale-out) to address the scalability problem by building large-scale distributed systems on top of commodity PCs.

This brings the following key benefits:

Lower upfront cost. A new service only needs to set up a few nodes to meet its initial user demands on the early stage.Linear scale. By adding more nodes, the system capacity can increase linearly. This means as the number of users grows, system capacity can be increased on-demand by adding more nodes with linear cost.

Some famous systems are designed and run to offer great horizontal scalability. These includes:

• Google File System (storage)
• Google’s BigTable (key-value store)
• Google’s MapReduce (computation)
• Hadoop with some Google’s system open-source counterparts, including HDFS, HBase, Map
• ReduceCassandra (key-value store)

The key technology that enables these systems is sharding — a way that partitions global system state to multiple relatively independent sub-states (shards) and builds a cluster with multiple nodes that process each sub-state in parallel.

With a good sharding design, the system capacity can be increased by adding more shards and nodes in the cluster linearly.

Scalability in Decentralized World

Given the scalability history from the centralized world, we believe that blockchain scalability can benefit from the aforementioned lessons (note that we only discuss on-chain solutions):

Currently, most existing on-chain solutions are to increase single blockchain performance, which we refer to as vertical scalability solutions.

For example, BTC has 600s block interval and 1M block size. LTC improves BTC by reducing block interval from 600s to 150s, and BCH improves BTC by increasing 1M block size to 8M.

However, because of global P2P network propagation limit, further decreasing block interval and/or increasing block size will increase stale block probability and thus waste hash power.To further increase single blockchain capacity, latest blockchains use improved consensus and new consensus algorithms such as PoS and delegated PoS (dPoS).

Two notable improvements are Ethereum’s GHOST implementation and EOS’s dPoS, where EOS’s dPoS only allows 21 nodes (or block producers) to produce a block in the chain.

Similar to centralized systems, we believe that blockchain vertical scalability is approaching its limit due to physical constraints of the P2P network and a full node.

To build a blockchain that meets global-wise commercial needs, we believe in horizontal scalability, which already demonstrates its success in centralized systems.

The basic idea of blockchain horizontal scalability is to partition single blockchain state to multiple blockchain states, i.e., shards. Each blockchain (a shard) could process transactions relatively independently, and each shard could interact with another shard seamlessly.

By increasing the number of shards in the network, the blockchain network throughput increases linearly.

QuarkChain promotes blockchain horizontal scalability technology (sharding) to build a blockchain that everyone can use anytime anywhere. In the following posts, we will elaborate the details of QuarkChain technology and the corresponding inspirations from the centralized counterparts.

The topics include:

• How QuarkChain partitions system state?
• How QuarkChain develops the consensus to drive all shards in a decentralized way while ensures the security in terms of various attacks such as double-spending attack, blacklisting attack, etc?
• How QuarkChain performs cross-shard transactions in an efficient and safe way?
• How QuarkChain supports smart contracts?
• How QuarkChain replaces a super-full node by a cluster of nodes?

In addition, we will also list some open questions we are facing in QuarkChain network and discuss potential directions to improve the system. We sincerely welcome any constructive feedback and suggestions in the journey.

About QuarkChain

QuarkChain aims to address blockchain scalability problem by utilizing horizontal scalability technologies. The mission of QuarkChain is to allow everyone in the world to use blockchain technologies anytime anywhere.