How Does Blockchain Technology Work?

The first blockchain was conceptualized by an individual (or group of individuals) known as Satoshi Nakamoto in 2008. It was launched the following year in January 2009 by Nakamoto as a core component of the cryptocurrency Bitcoin, where it serves as the public ledger for all transactions on the network. Through the use of a blockchain, Bitcoin became the first digital currency to solve the double spending problem without requiring a trusted authority and has been the inspiration for many additional applications. I believe that in one hundred years, blockchains will be as common and necessary as electricity is today. They will be fundamental pieces of the economy which nearly everyone will interact with on a daily basis. They will be so normal that we will forget they exist. We should expect that, over the next couple decades, we will see a Cambrian explosion of blockchain applications and organizations much like what happened with the internet over the past few decades or electrification in the early 20th century. If that’s true, it’s worth developing a basic understanding for blockchains, including why they matter and how they work. Why Blockchains Matter “It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking [about] what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them.” — Alfred North Whitehead In a blockchain, each time a transaction takes place, such as one party sending bitcoin directly to another, the details of that deal — including its source, destination, and date/timestamp — are added to what is referred to as a block. Using cryptography, blockchains dramatically increases , the ability for humans to collaborate with increasingly large numbers of other humans, a necessary feature of civilization. social scalability Historically, humans only transacted with members of a small tribe, often based on kinship alone, because there was no way for them to trust individuals outside that group. A wide variety of innovations over the last few millennia have increased social scalability by allowing humans to cooperate in larger groups. The modern legal system which lowers vulnerability to violence, theft and fraud allowing parties or individuals that have no pre-existing relationships to interact. The internet, particularly through rating systems, facilitated transactions between individuals with little or no social relationship. Blockchains have the potential to increase social scalability to a level that no previous technology has.For the first time, we have a way for one Internet user to transfer a unique piece of digital property to another internet user safely and securely without relying on a trusted third party. It’s hard to overstate how big of a deal this is. Think about digital contracts, digital keys (to physical locks or online locks), and digital ownership of physical assets including cars, houses, stocks, bonds and, of course, digital money. Note: There is a lot more to be said about Why Blockchains Matter. For the sake of not making this article insanely long, I will focus this article mostly on the “how” rather than the “why.” If you would like to read more about why blockchain technology matters check out Why is Bitcoin and Cryptocurrency Technology Valuable? , Money, blockchains, and social scalability , A Letter to Jamie Dimon , Beyond the Bitcoin Bubble and Why Decentralization Matters . How Does Blockchan Technology Work? Here’s the short jargon-y version: A blockchain is a ledger that uses cryptography and incentives to record transactions in a tamper-evident way. This allows for trust-minimized transactions between pseudonymous parties without requiring a trusted intermediary. Now, let’s unpack the pieces in there so that by the end of the article that will make perfect sense to you. Primitives Before we dive into blockchains, we need to establish a few primitives. Primitives are like building material. If you want to build a shed, you’re going to need some wood, some screws, a drill, and a saw. It would be good to get a brief idea on how each of those worked before you started building. That’s all we’re going to do here. The building blocks of blockchains are — surprise, surprise — primitives. cryptographic Cryptography is the study of techniques for communicating securely in the presence of third parties (AKA adversaries). Historically, a lot of cryptographic research was done by the military to ensure they could communicate securely without having their enemies be able to read their telegram or radio messages. Modern cryptography is used on every popular website, in credit card chips, and computer passwords. In all these cases, it’s trying to keep a third party (read: hacker, thief, etc.) from stealing sensitive data like your credit card information. The first building block we’ll start with is hash functions. Hash Functions A hash function is a mathematical function that lets you encrypt data.[1] A hash function has three properties: Its input can be any size. E.g. I could use the phrase “I like ice cream” as input or I could use the entire text of and both would work. War and Peace Using any size input, it comes up with a fixed size output determined by which particular hashing algorithm is being used. In order to keep it simple, we’ll use a 256-bit output since that’s what Bitcoin, the first blockchain, uses. Whether I entered the phrase “I like ice cream” as an input or I used the entire text of as the input, both would produce an output of the same size. War and Peace , the one used by Bitcoin, the phrase “I like ice cream” (no quotes) creates the output: Using the SHA 256 algorithm 138F4504A873C01D0864343FAD3027F03CA9BEA2F0109005FA4FC8C7DCC12634 The entirety of (all 587,287 words) copied from project Gutenberg produces the output: War and Peace 57027F5B3877ABBE43ACB101D59E963BC2CC0E86A6EBCCC34DFD035BBF83B096 Even though the input is 146,821 times longer, you get the same size output. Pretty cool and, it turns out, useful. More on that later. It is efficiently computable. For any given input string, you can figure out the output in a reasonable amount of time. Even if I want to use the entire text of as my input, it can still convert that into an output in a reasonable amount of time. War and Peace The above properties are for a general hash function. General hash functions let you map a large space of things to a small space which can be helpful for storing and working with different types of data. However, we’re going to focus on cryptographic hash functions since those are the ones used in blockchains. A cryptographic hash function requires three additional properties: Collision Resistance Deterministic Hiding Puzzle Friendliness Cryptographic Hash Functions Collision Resistance A collision is when two different inputs produce the same output. While the output size will be the same, the output itself is not going to be identical to other outputs. “I like ice cream” and the entire text of produce outputs of the same size, but the characters are different. War and Peace If they produced the exact same output and I sent you that encrypted text and you decrypted it, you wouldn’t know if I was telling you that I liked ice cream or that you should read . War and Peace It’s also notable that two “nearby” inputs do not result in nearby outputs. The hash of is completely different than the hash of War and Peace with 1 character changed. War and Peace Using SHA256, The entirety of produces the hash: War and Peace 57027F5B3877ABBE43ACB101D59E963BC2CC0E86A6EBCCC34DFD035BBF83B096 If I delete the final period, but leave everything else the same, it produces the hash: E2E2E4FFD582E20474F0310C2132EAE5F2D766C6A253C1BC4AF57861095B30FA This is very similar to how two “nearby” starting points in a very quickly lead to two completely different future trajectories. It is this “chaotic scrambling” property of hash functions which creates collision resistance. The outputs are appear random and so are unlikely to collide, but are able to map infinity to finite space, which allows us to compress data. chaotic system This particularly matters if you are sending around money. The public key to your wallet is produced from the private key using a cryptographic hash function. You don’t want someone else to be able to reverse engineer get your public key to figure out your private key (since they could then steal all your funds). A hash function is considered to be collision resistant if nobody can find a collision. In theory, we know they exist because the number of inputs is infinite, but the number of outputs is finite — there are only so many ways to arrange 64 characters. In practice though, the cryptographic hash function SHA 256 is collision resistant because no one has ever found a collision and the odds of it happening are astronomical. If every computer ever made by humanity had been trying to find a collision since the beginning of the universe, the odds of it happening are lower than the odds of the Earth being destroyed by a giant meteor in the next two seconds. So, it’s possible, but there are a lot of other, bigger problems we should worry about first. Deterministic Hiding Hashes look random but are actually deterministic. Their outputs are therefore reproducible so that as long as you use the same input, you always get the same output. This means two different parties who know the input can verify that the other person also knows the input by showing their hash to the other party. The second property that we want for a cryptographic hash function is hiding. Hiding means that if you are given the output, there is no feasible way to figure out what the input was. That is, there is no way for a third party observer to know that 138F4504A873C01D0864343FAD3027F03CA9BEA2F0109005FA4FC8C7DCC12634 means “I like ice cream.” However if I know the input is “I like ice cream” then I can easily verify whether or not you know that by checking your hash. However, if anyone else sees the hash, they cannot use it to figure out the input. This allows those who know the input to communicate securely about it in a public channel using the hashed output without worrying that a third party observer can figure it out. Puzzle Friendliness The third property we need is that they are puzzle-friendly. This means that if someone wanted to generate a hash that came up with the same output as “I like ice cream,” it’s extremely difficult to find another value that exactly hits this target. The algorithm to generate the hash is to try every possible string, ordered by length and then by alphabetical order, until you get one that hashes to the same value. There’s a very large haystack to search for a very small needle so we get probabilities that suggest an asteroid annihilating all life on Earth is a more pressing problem. Another way of saying this is that . This is the main thing which makes a cryptographic hash as compared to a hash used in data structures or a compression algorithm. cryptographic hash functions are hard to reverse This is also the arms race part of blockchains — hackers are always getting better at reversing hash functions so cryptographers are always having to come up with better hash functions. Having this quality is important because the blockchain is a “tamper-evident ledger,” a tamper proof record of transactions. The puzzle friendliness property is what makes it tamper-evident. If I wanted to replace an entry like “I like ice cream,” it should be impossible for me to do so without other participants being able to know. Using a good cryptographic hash function, even slight changes in the input will results in dramatically different outputs making the input tamper-resistant. Cryptographic hash functions look random (and so lack collisions), can’t be reversed, and hide inputs, but, at the same time, are completely deterministic. This is what makes them useful! This covers the basics of cryptographic hash functions. In the next section, we’ll look at how these hash functions are used to build blockchains. Transitive Hash Functions within the Blockchain The previous section about about hash functions above is really just a big introduction to how to hash a single piece of information like words in a book or a password. What if you wanted to hash collections of data like a bunch of documents or a sequence of transactions? Let’s start with a linear sequence of things you might want to consecutively hash. You start with some piece of data, A, and you hash it to get A#. Now you want to “mix in” B. So you just add B to it (A# + B) and take the hash of the whole thing. Now you have a new hash (B#) which has some information about both A (from the hash A# you used) and B. What you need is a hash function for which changing anything about A or B would result in a different final hash. This is a . transitive hash function A blockchain is a chain of hashes. When people talk about blockchains, one of the words that gets thrown around a lot is “trust-minimized.” Hashing in general, and in particular, are a core component of the trust-minimized property of blockchains. A hash pointer is just a hash that points at some other data structure like a previous block. hash pointers This is a big deal because it makes easy to see if someone tampers with the data. If we had a blockchain of 100 blocks and the tenth block contained the hash of the entire manuscript to and I changed one letter in the manuscript, it would change the hash pointer in that block and all subsequent blocks.[2] War and Peace Hash pointers ensure the integrity of the ledger (i.e. that it hasn’t been modified). Blockchains have been called because this hashpointer data structure is another, harder-to-forge way of making sure the ledger reflects reality. triple entry accounting For example, if there are two people collaborating on a project and one person tries to go back and modify something so that they make more money, it is immediately clear to their partner that they are trying to cheat — they will end up with a resulting hash that’s different. Merkle Trees Now what if you want to hash not just a linear sequence, but a sequence of sets like Bitcoin’s set of transactions in its sequence of blocks. The new thing you need is a rule for how to traverse each set of data, reducing the problem again to encrypting a consecutive series of things (A#, B#, etc.) A Merkle Tree is exactly this. It allows you to group blocks of data and hash them together into a structure that looks like, well, a tree branch with leaves. As with a blockchain, as long as we know the hash point at the “root” of the tree, then we can know that none of the data in the tree has been tampered with. If one bit, in one transaction is changed, then it changes every subsequent hash and the tampering is revealed. Using a Merkle tree is preferable over just a digest of all transactions because it allows for a much quicker and simpler test of whether a particular transaction is included in the set of all transactions. If you’re looking for more about blockchain and cryptocurrency for business, (as well as updates about future articles) click here for some other resources I’ve found valuable Blockchains We started by looking at hash functions. Then saw how we could add three properties: collision resistance, deterministic hiding and puzzle friendliness to make a hash function into a cryptographic hash function. Then, we saw how we could connect cryptographic hash functions together to make a transitive cryptographic hash function. When we add a Merkle tree to a transitive cryptographic hash function, we get a blockchain. Each block contains a set of “merkleized” transactions and the blocks are chained together. Proof of Work Blockchains The innovation of Satoshi Nakomoto, the anonymous creator(s) of Bitcoin, was to make one more rule. When you add a new set of transactions (AKA a block) into the existing chain, you have to solve a difficult and un-gameable problem. This is what so-called “miners” do. They try to solve a very difficult math problem and when they do, they receive a reward and hash the difficulty of the problem into the chain. This hash of the difficulty is a “proof of work,” which is a proof of how much computational energy went into trying to solve the problem. Among cryptographers, the invention of this “proof of work blockchain” was the biggest breakthrough by Satoshi Nakamoto. As a proof of work blockchain like Bitcoin becomes more valuable, miners make more money which incentives more miners to participate in the network which increases the difficulty of the problem. Anyone can download the blockchain on their computer and: Verify consecutively that every block transitively hashes correctly on top of the prior block. and They can calculate the total number of calculations required to have produced this data by reading the difficulty of each block and verifying the miners’ solution. The blockchain with the most accumulated proof of work (i.e. the most difficult and expensive to create) is considered the true blockchain. If someone handed you today’s Bitcoin blockchain on a disk you could very quickly estimate the huge amount of energy that was invested to construct this data. The benefit of this is that no one can “fake” a blockchain the size of Bitcoin with the same difficulties hashed into its blocks without spending the same amount of computational energy. As of mid-2018, it would to rewrite the Bitcoin blockchain from scratch. cost billions of dollars to buy enough computers and electricity Note: (Mining, Proof of Work, Proof of Stake and other consensus algorithms merit an entirely different post, but I wanted to at least mention them here in the context of blockchains.) Digital Signatures (AKA Private/Public Key Cryptography) A digital signature is the final primitive of blockchains. A successful digital signature scheme works more or less the same as a handwritten signature with two core properties. The first is verifiability. If I sign a transaction with my private key, someone else must be able to validate it using my public key. Likewise, only you can write your own signature, but anyone can see that it’s yours. A private key and a public key together form a key pair. The private key is something you don’t share with anyone, but use to sign transactions proving that it’s yours. It’s kind of like the wax seal that Harry Potter characters used to put on letters to prove that they sent a letter and that it wasn’t tampered with. You wouldn’t want to share the seal with anyone because then they could send messages pretending to be you. The public key is something you can give to everyone so they can verify that it’s your signature without them being able to sign it. This is like the recipient having a painting of your seal that they could confirm matches up correctly without actually being able to use it to stamp a message. Your public key is a hash of your private key. Your address, as in “What’s your bitcoin address so I can pay you?” is a hash of your public key. Private key > Public Key > Address hash hash Because of the properties of cryptographic hash functions, this means you can publish your address anywhere so that people can send you bitcoin, ethereum or any other cryptocurrency without being able to reverse engineer it to figure out your private keys. The second property is unforgeability. Your signature is tied to a particular document. You don’t want someone to be able to “snip” your signature off one document and glue it to the bottom of another one (like one saying you owe them money!). Bitcoin uses a digital signature scheme called the Elliptic Curve Digital Signature Algorithm to achieve these two properties. Other blockchains and cryptocurrencies use different digital signature schemes, but all have these same two properties of verifiability and unforgeability. Blockchains are like Books…. One of the easiest ways I’ve found to think about blockchains is like pages in a book. A book is a chain of pages with: Data — (AKA the text) on each page. Metadata — Metadata is information about itself. At the top or bottom of each page, there is some information about the book like the book title, the chapter title, and a page number. A blockchain is a chain of blocks with Data — the contents of the block Metadata — data about the block, including a reference to the previous block and a hash of the data contained in this block. If you ripped all the pages out of a book, you could put them back in order where the story made sense using the page number. Likewise, in a blockchain, each block references the previous block’s hash making it possible to validate in which order the transactions took place. You would know if someone scrambled them up just like you would know if someone scrambled up the page of a book. The hash is actually better than a page number because it contains a hash of all the data on the page. In theory, someone could rip the pages out of a book and white out and change the page numbers and you wouldn’t know. If someone tried to do the equivalent in our blockchain “book,” it would be transparent because changing one letter on a page would change the hash. This boils down to the fact that a blockchain is a sequence of transitively hashed data. Changing any one bit anywhere in the past would change everything after that in a chaotic and easy to detect way. Blockchain Explained: The Evolution of Blockchain Technology Now, let’s take these primitives and look at how they can be use to build a simple blockchain. These types of blockchains are the basis for cryptocurrencies like Bitcoin and Ethereum. By understanding how they work, you’ll have a foundation for understanding all the different blockchain and cryptocurrency projects. Goofy Coin GoofyCoin is the simplest blockchain we can imagine. There are three rules of GoofyCoin. The coin has a unique coin ID which Goofy then signs with his key so that anyone can use Goofy’s public key to confirm that the coin is a valid GoofyCoin. Goofy can create new coins whenever he wants and these newly created coins belong to him. “signed by pkGoofy” means Goofy signed the transaction (in this case the creation of a new coin) using his public/private keypair. If Goofy wants to transfer a coin to Alice, he creates a transaction saying “Pay this coin to Alice”. Because coins can only be sent using cryptographic operations, “this” is a hash pointer that points to the coin in question. Goofy then signs this transaction using his secret key making it a valid transaction. Anyone can check that it is a valid transaction by confirming Goofy’s signature. Whoever owns a coin can transfer it to someone else using cryptographic operations. In the example above, anyone can verify that Goofy created a valid coin and that he made a valid transfer to Alice. However, there’s a security problem with GoofyCoin. Let’s look at an example. With her new GoofyCoin, Alice could sign a transaction paying Bob for a sandwich. Anyone can verify the validity of a coin and who owns it by following the chain of hash pointers (the blockchain) back to its creation by Goofy, making sure all the hashes are correct along the way. Looks good right? Anyone can come in and see that these transactions are all valid. But, what if Alice uses that same coin to pay Chuck for a milkshake? Both Bob and Chuck think they own a GoofyCoin now. It looks like a valid transaction because both Chuck and Bob can see Alice signed it and follow it back on the blockchain to see Goofy created it as a valid coin. This is called a double-spending attack. Alice is spending the same coin twice which is not how we want money to work. You can’t give the same $10 bill to one person for a sandwich and to another person for a milkshake. A double-spending attack is one of the main problems which a functional cryptocurrency protects against. To solve the double spending problem, we’ll need to create a new coin called ScroogeCoin. ScroogeCoin ScroogeCoin works just like GoofyCoin with a few different tweaks. Unlike in Goofy Coin, Scrooge publishes a blockchain with a history of all transactions. Scrooge will sign the hash pointer of each block in the blockchain. Anybody can then verify that a transaction is valid by checking Scrooge’s signature on the block that records the transaction. Because Scrooge is looking over every transaction, he can make sure nobody double spends. If Alice uses a coin to buy a sandwich from Bob, Scrooge will make sure that she can not also use that same coin to buy a milkshake from Chuck by refusing to validate that transaction. Because all the blocks are connected by hash pointers into a blockchain, Alice, Bob, Chuck and anyone else using ScroogeCoin can monitor that Scrooge isn’t going back to change any data. Though they couldn’t stop Scrooge from doing it since he is the one in control of the blockchain, they would at least know that he did it. If a year ago, Alice had paid 1 ScroogeCoin to Bob for a sandwich and Scrooge went back and changed it to say she paid 10 ScroogeCoins, it would change the hash pointers of every single subsequent block making it easy to know that Scrooge had tampered with the data. This is the puzzle-friendly property we looked at earlier. This makes it easier to trust that the data is correct, rather than if these were just records kept in a large spreadsheet only controlled by Scrooge where he could go back and change one of the previous entries to give himself more money.[3] ScroogeCoin works in the sense that everyone can see which coins are valid. And, as long as Scrooge monitors all the transactions and rejects attempts to double spend, there is no double spending problem. All the participants can confirm that Scrooge is preventing double spending by looking at the blockchain to confirm that all transactions are valid and no coin has been spent twice. The potential problem with ScroogeCoin is Scrooge himself. If Scrooge wants to, he can make himself rich by demanding that everyone who does a transaction pays him a large fee for doing the transaction, or he could create a bunch of ScroogeCoins for himself, or he could just get bored and stop verifying the transactions. The problem, in short, is centralization. Although Scrooge is happy with the system, the users might not be. This system is not unlike the one we have today. You could replace “Scrooge” with “Citibank” and this is not unlike how the existing banking system works. We trust banks or other financial intermediaries to not lie about transactions. For the most part, this works pretty well. These institutions are regulated by federal authorities and subject to some level of competition from other firms which encourages them to be honest and efficient. Many people, particularly those that have spent their whole lives in developed countries, don’t see much value in blockchain because the existing system works pretty well. We have a legal system designed to prosecute anyone that violates their contracts and businesses are incentivized to protect their reputation by behaving honestly. However, there are situations where this breaks down. Hyperinflation, when a country prints new money devaluting it’s existing money supply has happened many times in history including Weimar Germany in the 1930s, Argentina and Zimbabwe in the early 2000’s, and Venezuela in the 2010’s. In all of these cases, it was the equivalent of Scrooge deciding to make more Scrooge coins for himself. The same scenario can happen in other types of coordination scenarios. For example, your company is working on a joint construction project with four other companies and one of them tries to lie about how much work they did to earn a larger percentage of the contract. These are all scenarios where decentralized blockchains have the potential to help. A decentralized blockchain means that instead of a single party controlling it, the ledger is distributed among different parties. Each of these parties maintain their own copy of the ledger. Because of the cryptographic properties we talked about above, if one party tries to tamper with the ledger, it will immediately be clear to all the other parties that someone tried to go Scrooge and cheat. How Blockchains Achieve Decentralization: Public (AKA Permissionless) vs. Private (AKA Permissioned) Blockchains There are two main ways in which a blockchain can go from centralized (e.g. ScroogeCoin) to decentralized. Private Blockchains (AKA Permissioned blockchains) Public Blockchains (AKA Permissionless blockchains) The difference between public and private blockchain is related to who is allowed to participate in the network. A public blockchain network is completely open and anyone can join and participate. Bitcoin and Ethereum are examples of public blockchains. A private blockchain network requires an invitation. It is a permissioned network that places restrictions on who is allowed to participate in the network and in which transactions. Some examples of private blockchains are MultiChain, R3’s Corda, Hyperledger Fabric and Chain Core. The difference in who is allowed to participate matters because it affects how decentralization is achieved. In our example of ScroogeCoin, it suffers from the same problem where a single controls things. trusted third party Public blockchains are secured by cryptoeconomics and cryptocurrency mining. (The jargon-y explanation for this is that they use a combination of economic incentives, game theory, cryptography, and computer science to agree on the current state of the ledger — these are topics for a future article). A permissioned or private blockchain is a blockchain where the consensus process (i.e. Scrooge’s job) is controlled by a pre-selected set of actors (e.g. individuals, companies, members of a consortium). For example, you could have 15 banks each of which have permission to write to the ledger and each of which must sign every block in order for the block to be valid. This is more decentralized than having a single party in charge of validating all the transactions, which has a few advantages. Advantages and Examples of Private Blockchains The primary reason companies may choose to use a private blockchain is to increase the transparency and trust of their processes with their partners in the network. Because the blockchain functions as a tamper-evident log, each company (for example a group of 15 banks) can trust that no one tampered with pre-existing data. Compared to their public counterparts, private blockchains are also more scalable. You may have heard about with public blockchains like Bitcoin and Ethereum. These are a result of trying to scale the blockchain while also keeping it decentralized and safe in an open environment where anyone can join. scalability problems Because private blockchains happen in a closed environment, they can prevent attacks by controlling who gets to join the network. As an example of a private blockchain, let’s look at Everledger which is working with the diamond industry to help reduce fraud and improve insurance. The diamond industry has high levels of theft because diamonds are small. There is also a lot of fraud because of document tampering (lying about the quality or size of diamonds) and synthetic diamonds being used in lieu of real diamonds. There are also ethical issues with so-called “blood diamonds,” and customers want to know they are buying an authentic, conflict-free gem. About $2 billion is lost from jewelry fraud every year so there’s a meaningfully sized problem to solve here. Everledger works with the certificate houses which certify the diamonds. Each diamond is measured on forty different criteria and then assigned a unique identifier. That information is then inserted into the blockchain where it is tamper-evident.[4] Every time the diamond is sold or transferred to another player in the supply chain (E.g. from the mining company to the company that cuts and polishes it), that transfer is likewise recorded in the blockchain. That makes it possible for an end-consumer to know exactly where their diamond was mined and every company that touched it across the supply chain. There can be more than a dozen different stakeholders through the life of a diamond. If you want more details, You can watch Leanne explain Everledger’s business in detail . It also makes it possible for insurers to know they are paying legitimate claims. Another example of a private blockchain is Construtivo, a Brazilian software company which for the past fifteen years has managed project data for large infrastructure construction projects. They help make sure that all stakeholders have a consistent view of a project’s status and progress. To satisfy their customers’ desire for greater transparency and auditability, Construtivo are using a private blockchain with each stakeholder maintaining their own copy of the ledger, so each stakeholder can be sure none of the other companies are short changing them. The Case Against Private Blockchains One of the common criticisms of private blockchains is that they are “just shared databases.” That is, a private blockchains is basically just like having a shared Google Docs spreadsheet where you are trusting no one else to go back in and change the data entered. Private blockchains claim to protect data by controlling who gets to join the network, but they are still vulnerable to the same type of network hacks we’ve seen with private corporations like and . Yahoo Equifax Public blockchains don’t have this problem because everything is public, meaning there’s no network to hack into. Consensus isn’t driven by the ability to get into the network; it’s driven by who spends real world energy in a way which is impossible to fake and which makes the blockchain much more expensive to “hack” (as discussed in the Proof of Work section above). Blockchains, both public and private, suffer from the need for users to protect their private keys. And users constantly fail to do it correctly, which is why individuals get hacked — lots of people have had their bitcoin stolen — even if the system does not. So, the argument against private blockchains is that they rely on network security to restrict bad actors from joining the network. Detractors argue that this replaces the best, most novel part of blockchain security (proof of work) with the worst part (users protecting their private keys). Advantages and Examples of Public Blockchains Public blockchains are likely the ones you’ve heard the most about and include networks such as Bitcoin and Ethereum. One of the biggest advantages of public blockchains is related to the value of open (AKA private AKA permissioned) versus closed (AKA public AKA permissionaless) networks. The easiest comparison is closed networks versus the internet in the 90’s. At first, there were a number of companies operating closed/permissioned networks like Compuserve and Prodigy. They linked computers to a central server, with news and other information, but not to the broader Internet. The idea was more appealing to large companies because it seemed safer. Companies wouldn’t be exposed to the whole internet, but would be able to retain control over what could be put on the network. The internet won out over closed systems like Compuserve and Prodigy because there are 100x more developers building on the internet than on any single intranet. Today, there are maybe 20 million good developers in the world and the largest companies (Facebook, Google, and Amazon) employ, at most, a few tens of thousands. That means 99% of the best developers in the world always work for someone else. It is better to create an ecosystem that gets all the world’s smartest people toiling for your goals. If you rely solely on your own employees, you’ll never solve all your customers’ needs. The other primary advantage of public blockchains (at least in theory) is that they are more decentralized and thus more . censorship resistant ScroogeCoin is the least censorship resistant system you can imagine. If Scrooge decides he doesn’t want to let you use the network, he can do that. Private blockchains are more censorship resistant. If you have a private blockchain where there are 5 validators, then you need three of them to collude and lie about something which is more difficult than just a single person. If you have a public blockchain with 5000 validators, then it is much harder for them to collude. You would have to bribe, coerce or otherwise convince 2501 actors to collude, which is much more challenging than convincing a single person or small group of individuals. The nature of public blockchains allows anyone to join in the security of the network, which has resulted in the . world’s largest supercomputer Because they are public, many different companies are building products on top of them which . makes them more valuable In 2017, there was a wave of other crypto networks launched on the Ethereum blockchain. These networks did not need to ask for permission to launch on Ethereum from the Ethereum team. They just had to build their network and hook it up to the open Ethereum protocol. This increased the value of the Ethereum network by over 100x. This sort of wave of innovation would have been impossible if Ethereum was a private blockchain that required getting individual permission for each project. One of these projects is called OmiseGO which is trying to help people send remittance payments. For people who work in one country and send money back to their families in another country, current solutions can be extremely expensive. Because OmiseGO knew Ethereum was a public blockchain, they could safely invest in building on top of Ethereum knowing there was no risk of Ethereum being able to shut them down in the future. This is not the same as building on top of private blockchain where it is easier to be censored. If you’re looking for more about blockchain and cryptocurrency for business, (as well as updates about future articles) click here for some other resources I’ve found valuable Conclusion We started with why blockchains matters in the first place: their ability to allow large groups of individuals to coordinate with each other. Then we explored blockchain primitives, or cryptographic hash functions, which have three important properties: Collision Resistance — Two similar strings of text produce very different outputs. If you change just a single character in the manuscript of , you get dramatically different results War and Peace Deterministic Hiding — There is no way for a third party observer to know that 138F4504A873C01D0864343FAD3027F03CA9BEA2F0109005FA4FC8C7DCC12634 means “I like ice cream.” Puzzle Friendliness — If someone wanted to generate a hash that came up with the same output as “I like ice cream,” it’s extremely difficult to find another value that exactly hits this target. We then examined how hash functions can be strung together into a transitive hash function. When we add a Merkle tree to a transitive cryptographic hash function, we get a blockchain. Each block contains a set of “merkleized” transactions and the blocks are chained together. We then saw how bitcoin uses a technology called proof of work to secure the blockchain in a way that is impossible to fake. Next, we investigated Goofycoin and Scroogecoin to see how blockchain technology had evolved over time to solve the double spending problem Finally we looked at how public and private blockchains try to achieve decentralization. Blockchains are still a very young technology. They’ve only been around since 2009 and we should expect that over the coming decades, we will see an explosion of different blockchain applications and networks. Already, there are thousands of different projects working on modifying the technology to see if and how it can be improved or adjusted for specific use cases. You now have an understanding of the fundamentals behind blockchain technology and how blockchains work. Most, if not all, of the biggest innovations in the next generation of blockchain technology haven’t yet happened. There will never be a better time in history to start learning and get involved. If you’re interested in learning more, sign up for my newsletter and I’ll send you a list of my top 10 resources for learning more about blockchain, a toolkit of blockchain resources, and notifications about future articles I publish on blockchain technology. Acknowledgements: Many of the images and ideas in this article are from the Coursera course Bitcoin and Cyrptocurrency Technologies developed by Arvind Narayanan and offered for free. I would recommend. I also read and would recommend the companion text book . All images credit to them as well unless otherwise noted. Thanks to Gary Basin, Dhruv Bansal, Joe Kelly, and Juvoni Beckford for their extensive feedback on early drafts. Footnotes [1] Another way of saying this is that it’s just a way of mapping one text A to another one B in a way that makes it hard to rederive the the A if all you have is the B and also where very few different texts C, D, E, etc will generate the same output when passed through the same function (minimal tendency to create “collisions”) [2] (because the hash is collision resistant — two different inputs effectively never produce the same output) [3] For the simplicity of explanation, each block in ScroogeCoin only has one transaction, but in practice you would put multiple transactions in one block and Scrooge would just reject any double spend attempts within the block. [4] When you are dealing with tracking real-world events or items as opposed to digital ones, you necessarily re-introduce the scrooge problem. In the case of Everledger, we’re dependent on some Scrooge-like entity to enter data at every step of the way (known as the oracle problem — you need to trust some “oracle” to get the data “on chain.”) Once that data is on chain though, it becomes tamper-evident.) Taylor Pearson is the author of The End of Jobs and writes about entrepreneurship, history, complexity and blockchain technologies at TaylorPearson.me . Sign up to receive his popular newsletter and updates .