Beyond Sharding and Rollups: A New Approach to Blockchain Scalability

Vitalik Buterin has produced yet another well-argued piece on why one Ethereum scalability approach (ZK rollups) is better than others (Ethereum sharding). I agree with almost everything Vitalik has ever written on the topic and yet I fundamentally disagree with the vision he paints. The reason is that his arguments fail to address the fundamental issue: Blockchains’ inherent structure is the real obstacle to scalability. As long as the business layer, which provides value and experience, is coupled to the network layer, which provides scalability, we will not be able to provide value at scale. Scalability - Blockchain’s great dilemma dilemma (noun): a choice between two (or, loosely, several) alternatives, which are or appear equally unfavorable - oed Public blockchain ecosystems like Ethereum have a problem. The monolithic “L1” mainnets are neither suitable nor sufficiently performant and cheap for all the use cases that are envisioned. This problem is usually condensed into talking about “blockchain scalability”. This problem is serious. Vitalik Buterin, co-founder of Ethereum, has been writing about “blockchain scalability” for over 10 years. Many solution approaches have been proposed and implemented - Lightning, Sharding, Optimistic Rollups, ZK Rollups, Plasmas, Validiums, Baselining, Subnets, Parachains, Appchains... The pros and cons of approaches are hotly debated. Choosing the right option out of the above is a real dilemma. The problem is real, ecosystems like Ethereum must make a choice. But all of the above choices are equally unfavorable. As with every good dilemma, arguments can be made as to why one choice is better than another. The Ethereum ecosystem is aligning around rollups, and Vitalik Buterin argues that long term this will converge to ZK rollups. His arguments are strong, revolving around the advantages of heterogeneity and developer independence as well as the technical advantages of data efficient proofs. But in the same piece of writing, he also points out that all the options under consideration are all essentially the same. "Layer 2s" and "sharding" often get described in public discourse as being two opposite strategies for how to scale a blockchain. But when you look at the underlying technology, there is a puzzle: the actual underlying approaches to scaling are exactly the same. You have some kind of data sharding… The main difference is: who is responsible for building and updating those pieces, and how much autonomy do they have? Vitalik Buterin And they are all the same in the sense that the shards/rollups/subnets are in a very real sense independent chains. The arguments about whether to use one type of rollup vs another are equivalent to arguing which type of glue is best suited to put back together the shards (pun intended) of something that is fundamentally broken. The way Vitalik talks about this is how to preserve the “feeling” that the ecosystem is one coherent thing, ie how best to hide the glued seams: While Ethereum branches out, the challenge is in preserving the fundamental property that it still all feels like "Ethereum", and has the network effects of being Ethereum rather than being N separate chains… Moving tokens from one layer 2 to another requires often centralized bridge platforms, and is complicated for the average user.Vitalik Buterin These problems are known, understood, and a lot of effort is being put into smoothing out these seams. I’ve already argued why bridges won’t cut it. But why try to glue something fundamentally broken in the first place and not design something that solves the fundamental problem? The Internet Analogy - Decoupling Business and Network Everyone in the space loves analogies with the internet and for good reasons: The internet works. It provides connectivity between applications, assets, and users, and it is pretty much infinitely scalable. It has many of the properties espoused by blockchain networks. Common Experience: It has the fundamental property of uniformly feeling like “the internet” and creating network effects across it. It doesn’t feel like the N peered tier 1/2/3 subnetworks that it really is. The browser provides this experience. Heterogeneity: Centralized controls and standards are minimal (TCP/IP, BGP, DNS, etc). It’s possible for developers to try almost anything new. Application operators are in complete control of their system—from data permissions to SLAs. Composability: Publicly facing APIs can be composed together into new experiences on top of the underlying apps. Scalability: Just add a new server or router to extend. One of the core design differences between the internet and blockchains like Ethereum is that the business layer is completely decoupled from the network layer. I’ve already pointed out above that the internet at the network level really is a glued-together patchwork. Traffic gets automagically (thanks to BGP) routed through independently controlled peered networks. We only experience this in any way when it goes wrong and a service like Facebook, which runs on its own subnetwork, disappears from the internet. The only thing that matters to you as a user is that you have sufficient connectivity to one of Facebook’s servers - the business layer of the internet. This works because Facebook and you have commonly agreed to allow your (hopefully TLS-encrypted) traffic to be routed through some third-party infrastructures (backbones, ISPs, etc). The business layer - authentication, reading and sending data, making a post, etc. - is just between you, the user, and Facebook as the application provider. Fully replicated blockchains like Ethereum, as well as the “sharding” techniques above, are fundamentally different. The network acts as the network and messaging channel (gossip protocols), data persistence layer (storing and serving blocks), logic execution/validation (smart contracts), ordering (mining), etc. It’s all baked together, which makes each network/shard a monolithic silo, a “virtual mainframe”. The business topology of the internet, meaning the connections between users and apps, as well as composability between apps, is forced to follow network topology. Ergo, sharding your network layer for scalability shards your business layer creating boundaries that are hard to cross, fragmented experiences, and ultimately new silos. Canton Network is designed from the ground up to decouple the business and network layers like the internet. The business layer, which is the thing we most often think of as “the blockchain” is entirely between the involved participants - the users’ and application providers’ “participant nodes”, which are analogous to servers on the internet. They make dynamic use of infrastructure, so-called “synchronizers”, which only takes care of transmitting and ordering (encrypted) messages, equivalent to routers, subnetworks, or ISPs on the internet. They do just enough to ensure deterministic execution, transaction atomicity, and double spend protection between the participants. This allows the business layer to grow seamlessly, organically, and without crude glue. No shards, no silos. For example, imagine we have some funds, cash, and a trading app each using only their dedicated synchronizer infrastructures and run by their respective operators (OP in the picture). Buyer and Seller would like to trade, but can’t as there is no common connectivity - effectively you have three apps, each running on their own LANs which Buyer and Seller happen to be connected into. With Canton, all you have to do is add a common synchronizer - the equivalent of a WAN or internet backbone. No need to bridge, no need to “move” tokens, and no need for the app/asset operators to change their application. All it takes for the participants is to connect to an additional common synchronizer. Once they’ve done so, they can coordinate the consensus needed to add an atomic transaction involving all three apps to “the chain”. Sure, you need to put some trust in this new infrastructure to perform its basic duties correctly, but trust for message ordering and delivery are solved problems. Choose a trusted centralized operator (ISP, backbone operator) or use a BFT algorithm (like a blockchain). And if in doubt, you can remove or switch the synchronizer as easily as you added it - without migrating your application to a new server. Conclusion The entire discourse about blockchain sharding and rollups is a debate about which remedy to a terminal disease is the least bad. The fundamental problem of blockchains like Ethereum is that they closely couple business topology with network topology. Scalable networks are inherently a patchwork - as demonstrated by the internet, the ability to add “patches” is what scalability is all about. But at the business level, such patchworks are antithetical to the uniform experience and universal composability we need to realize blockchain’s value proposition. Gluing together independent silos with messaging standards and bridges is not innovation. Networks like SWIFT have optimized that model to a high degree. What we need is a middle ground that is able to deliver the atomic composability and uniformity of L1 blockchains at the business level while offering the flexible network topology, subnetwork independence, and scalability of something like the internet. Vitalik Buterin has produced yet another well-argued piece on why one Ethereum scalability approach (ZK rollups) is better than others (Ethereum sharding). I agree with almost everything Vitalik has ever written on the topic and yet I fundamentally disagree with the vision he paints. The reason is that his arguments fail to address the fundamental issue: Blockchains’ inherent structure is the real obstacle to scalability. As long as the business layer, which provides value and experience, is coupled to the network layer, which provides scalability, we will not be able to provide value at scale. Vitalik Buterin another well-argued piece another well-argued piece Scalability - Blockchain’s great dilemma Scalability - Blockchain’s great dilemma dilemma (noun) : a choice between two (or, loosely, several) alternatives, which are or appear equally unfavorable - oed dilemma (noun) oed oed Public blockchain ecosystems like Ethereum have a problem. The monolithic “L1” mainnets are neither suitable nor sufficiently performant and cheap for all the use cases that are envisioned. This problem is usually condensed into talking about “blockchain scalability”. This problem is serious. Vitalik Buterin, co-founder of Ethereum, has been writing about “blockchain scalability” for over 10 years . Many solution approaches have been proposed and implemented - Lightning , Sharding , Optimistic Rollups , ZK Rollups , Plasmas , Validiums , Baselining , Subnets , Parachains , Appchains ... The pros and cons of approaches are hotly debated. for over 10 years for over 10 years Lightning Lightning Sharding Sharding Optimistic Rollups Optimistic Rollups ZK Rollups ZK Rollups Plasmas Plasmas Validiums Validiums Baselining Baselining Subnets Subnets Parachains Parachains Appchains Appchains Choosing the right option out of the above is a real dilemma . The problem is real, ecosystems like Ethereum must make a choice. But all of the above choices are equally unfavorable. real dilemma As with every good dilemma, arguments can be made as to why one choice is better than another. The Ethereum ecosystem is aligning around rollups, and Vitalik Buterin argues that long term this will converge to ZK rollups. His arguments are strong, revolving around the advantages of heterogeneity and developer independence as well as the technical advantages of data efficient proofs. argues argues But in the same piece of writing, he also points out that all the options under consideration are all essentially the same. "Layer 2s" and "sharding" often get described in public discourse as being two opposite strategies for how to scale a blockchain. But when you look at the underlying technology, there is a puzzle: the actual underlying approaches to scaling are exactly the same. You have some kind of data sharding… The main difference is: who is responsible for building and updating those pieces, and how much autonomy do they have? Vitalik Buterin "Layer 2s" and "sharding" often get described in public discourse as being two opposite strategies for how to scale a blockchain. But when you look at the underlying technology, there is a puzzle: the actual underlying approaches to scaling are exactly the same. You have some kind of data sharding… The main difference is: who is responsible for building and updating those pieces, and how much autonomy do they have? "Layer 2s" and "sharding" often get described in public discourse as being two opposite strategies for how to scale a blockchain. But when you look at the underlying technology, there is a puzzle: the actual underlying approaches to scaling are exactly the same. You have some kind of data sharding… The main difference is: who is responsible for building and updating those pieces, and how much autonomy do they have? Vitalik Buterin Vitalik Buterin Vitalik Buterin And they are all the same in the sense that the shards/rollups/subnets are in a very real sense independent chains. The arguments about whether to use one type of rollup vs another are equivalent to arguing which type of glue is best suited to put back together the shards (pun intended) of something that is fundamentally broken. The way Vitalik talks about this is how to preserve the “feeling” that the ecosystem is one coherent thing, ie how best to hide the glued seams: While Ethereum branches out, the challenge is in preserving the fundamental property that it still all feels like "Ethereum", and has the network effects of being Ethereum rather than being N separate chains… Moving tokens from one layer 2 to another requires often centralized bridge platforms, and is complicated for the average user. Vitalik Buterin While Ethereum branches out, the challenge is in preserving the fundamental property that it still all feels like "Ethereum", and has the network effects of being Ethereum rather than being N separate chains… While Ethereum branches out, the challenge is in preserving the fundamental property that it still all feels like "Ethereum", and has the network effects of being Ethereum rather than being N separate chains… Moving tokens from one layer 2 to another requires often centralized bridge platforms, and is complicated for the average user. Moving tokens from one layer 2 to another requires often centralized bridge platforms, and is complicated for the average user. Vitalik Buterin Vitalik Buterin Vitalik Buterin These problems are known, understood, and a lot of effort is being put into smoothing out these seams. I’ve already argued why bridges won’t cut it . But why try to glue something fundamentally broken in the first place and not design something that solves the fundamental problem? bridges won’t cut it bridges won’t cut it The Internet Analogy - Decoupling Business and Network Everyone in the space loves analogies with the internet and for good reasons: The internet works. It provides connectivity between applications, assets, and users, and it is pretty much infinitely scalable. It has many of the properties espoused by blockchain networks. Common Experience: It has the fundamental property of uniformly feeling like “the internet” and creating network effects across it. It doesn’t feel like the N peered tier 1/2/3 subnetworks that it really is. The browser provides this experience. Heterogeneity: Centralized controls and standards are minimal (TCP/IP, BGP, DNS, etc). It’s possible for developers to try almost anything new. Application operators are in complete control of their system—from data permissions to SLAs. Composability: Publicly facing APIs can be composed together into new experiences on top of the underlying apps. Scalability: Just add a new server or router to extend. Common Experience: It has the fundamental property of uniformly feeling like “the internet” and creating network effects across it. It doesn’t feel like the N peered tier 1/2/3 subnetworks that it really is. The browser provides this experience. Common Experience: Heterogeneity: Centralized controls and standards are minimal (TCP/IP, BGP, DNS, etc). It’s possible for developers to try almost anything new. Application operators are in complete control of their system—from data permissions to SLAs. Heterogeneity: Composability: Publicly facing APIs can be composed together into new experiences on top of the underlying apps. Composability: Scalability: Just add a new server or router to extend. Scalability: One of the core design differences between the internet and blockchains like Ethereum is that the business layer is completely decoupled from the network layer. I’ve already pointed out above that the internet at the network level really is a glued-together patchwork. Traffic gets automagically (thanks to BGP) routed through independently controlled peered networks. We only experience this in any way when it goes wrong and a service like Facebook, which runs on its own subnetwork, disappears from the internet . automagically disappears from the internet disappears from the internet The only thing that matters to you as a user is that you have sufficient connectivity to one of Facebook’s servers - the business layer of the internet. This works because Facebook and you have commonly agreed to allow your (hopefully TLS-encrypted) traffic to be routed through some third-party infrastructures (backbones, ISPs, etc). The business layer - authentication, reading and sending data, making a post, etc. - is just between you, the user, and Facebook as the application provider. Fully replicated blockchains like Ethereum, as well as the “sharding” techniques above, are fundamentally different. The network acts as the network and messaging channel (gossip protocols), data persistence layer (storing and serving blocks), logic execution/validation (smart contracts), ordering (mining), etc. It’s all baked together, which makes each network/shard a monolithic silo, a “virtual mainframe”. The business topology of the internet, meaning the connections between users and apps, as well as composability between apps, is forced to follow network topology. Ergo, sharding your network layer for scalability shards your business layer creating boundaries that are hard to cross, fragmented experiences, and ultimately new silos. Canton Network is designed from the ground up to decouple the business and network layers like the internet. The business layer, which is the thing we most often think of as “the blockchain” is entirely between the involved participants - the users’ and application providers’ “participant nodes”, which are analogous to servers on the internet. They make dynamic use of infrastructure, so-called “synchronizers”, which only takes care of transmitting and ordering (encrypted) messages, equivalent to routers, subnetworks, or ISPs on the internet. They do just enough to ensure deterministic execution, transaction atomicity, and double spend protection between the participants. This allows the business layer to grow seamlessly, organically, and without crude glue. No shards, no silos. For example, imagine we have some funds, cash, and a trading app each using only their dedicated synchronizer infrastructures and run by their respective operators (OP in the picture). Buyer and Seller would like to trade, but can’t as there is no common connectivity - effectively you have three apps, each running on their own LANs which Buyer and Seller happen to be connected into. With Canton, all you have to do is add a common synchronizer - the equivalent of a WAN or internet backbone. No need to bridge, no need to “move” tokens, and no need for the app/asset operators to change their application. All it takes for the participants is to connect to an additional common synchronizer. Once they’ve done so, they can coordinate the consensus needed to add an atomic transaction involving all three apps to “the chain”. Sure, you need to put some trust in this new infrastructure to perform its basic duties correctly, but trust for message ordering and delivery are solved problems. Choose a trusted centralized operator (ISP, backbone operator) or use a BFT algorithm (like a blockchain). And if in doubt, you can remove or switch the synchronizer as easily as you added it - without migrating your application to a new server. Conclusion The entire discourse about blockchain sharding and rollups is a debate about which remedy to a terminal disease is the least bad. The fundamental problem of blockchains like Ethereum is that they closely couple business topology with network topology. Scalable networks are inherently a patchwork - as demonstrated by the internet, the ability to add “patches” is what scalability is all about. But at the business level, such patchworks are antithetical to the uniform experience and universal composability we need to realize blockchain’s value proposition. Gluing together independent silos with messaging standards and bridges is not innovation. Networks like SWIFT have optimized that model to a high degree. What we need is a middle ground that is able to deliver the atomic composability and uniformity of L1 blockchains at the business level while offering the flexible network topology, subnetwork independence, and scalability of something like the internet.