Blockchain Development Through the Eyes of a Backend Developer

Good afternoon, everyone! I've been doing Backend development for quite a long time, and for the last few years, I've been writing more and more different blockchain projects (Solidity on EVM). Diving into blockchain was not easy for me, and my backend brain broke down several times, so I decided to share my view on making the switch to blockchain development.

Blockchain is a very cool technology that can move our world forward. But for now, a lot of people use it to buy-sell-swindle-fraud-boost. I will not and do not plan to consider crypto as an asset.

Yes, many different videos and posts say, “Here, such-and-such crypto can grow now, and such-and-such crypto has fallen. Guys, let's invest, let's buy...” I will not tell you anything about it.

I will only say one thing - you should avoid crypto for making money. If you are a developer, it is better to try to earn as a developer and not get involved in investments. And if you really want to invest, do not go into crypto.

About blockchain and why it's interesting

It's worth starting with an important note. Blockchain is not cryptocurrency. Blockchain is the technology on which cryptocurrency is built and calling cryptocurrency a blockchain is like calling the entire development industry Javascript. Yes, JavaScript seems to be a special development case, but when discussing development, we don't always mean JavaScript. Although some people only mean JavaScript...but that’s a story for another day.

Going to blockchain for the money

The first thing that comes to mind is money. Blockchain developers are paid quite well. I have personally opened such vacancies. I have personally responded to such vacancies, where you can get more than the same backend developer for the same amount of time spent per day at work. Blockchain developers are worth their weight in gold in blockchain-based startups. Especially a good blockchain developer!

Who is a good backend developer?

Becoming a good backend developer is impossible without ever dropping or fixing a prod. Maybe I'm just a loser and only learn through negative experiences, but that's the theory I have:

1. If the backend developer is experienced, then he has run solutions in prod.
2. If he has run it in Prod, then he knows how it runs and knows what happens to the product under load.
3. If he has kept services under load, then he has caught downtimes and service crashes.
4. And if he's caught service crashes, he's probably picked them up.

You can't exchange experience with failure, but the experience of failure lets you realize that you have experience. It's hard to become a good blockchain developer without losing money:

• If I haven't had money stolen from my contract yet, it means that I haven't launched something with real money (at least $1000 on my contract).
• If I haven't launched something with real money or only on test networks (with fake money), then I have no idea what awaits me in the toxic world of blockchain.
• If I don't know what's in store for me, I'll get burned on the first launch, and it's a question of the team/company that is willing to take the risk of being the first payer for my mistake.

Going into blockchain for the technology

The image above shows the Wright brothers' first airplane in the world, flying pretty badly. But it flew, and in its day, the average person's attitude to airplanes was something like this:

• Expensive
• Inconvenient
• Inconvenient, incomprehensible

And now, the airline industry is a wonderful part of our lives, connecting people all over the planet within a few hours. Logistics is now at a level the Wright brothers never dreamed of! The whole world lives differently because of airplanes.

I would say the same about blockchain now - it's expensive and inconvenient and unclear why. Until I was immersed in blockchain development, it looked to me like something useless for cheating investors (=hamsters). But if you look at it from the other side, it's possible to store any facts in a decentralized way without the possibility of tampering. The “without the possibility of tampering” is an important detail.

But alas, the associations with the word “blockchain” are rather dull and monotonous:

• Bitcoin
• Investment
• Scam
• ETH, Ripple, {insert coin name}
• Bubble that's about to burst

If we throw a bundle of paper money into the stove, the stove will burn well, and the money will even give heat for some time. But that's absurd.

And so it is with blockchain. Using it only for money and crypto is bad, but the other thing is not taking root yet...

One of the main drivers of any product development is money. If there is something on which to make good and powerful money, then this “something” will be actively developed. And that is why, so far, financial projects based on blockchain are truly based on making money for someone to make money and losing money for someone else.

Blockchain Development Theory

Decentralization

What is the difference between a centralized and decentralized service? Let's start with a centralized system. There's me and someone else, and we've decided to use a service between us, like a bank or another provider, to facilitate a transfer.

Let's imagine that we have a bank, which is a centralized service. I command this bank: “Please transfer 100 dollars to this person”. The bank records that I have 100 dollars less, and someone else has 100 dollars more.

But what is the problem with the centralized service? There is an owner behind this centralized service, isn't there? Usually, some large companies, holdings, it doesn't matter; in our case, let it be one person. This one person can say: “Let's do this. Let Alex send 100 dollars, but no one will receive these 100 dollars. I need them more”.

Centralized services have their owners. The problem is that the owner can make negative decisions, taking away money. And it can be not only about “taking the money for yourself”. For example, you can write, “We have 100,500 dollars in the bank,” and hope that all depositors will not go after that money... as happened with SVB and other banks that died.

Bitcoin was invented to decentralize money management.

A decentralized service is built on a network of nodes where each node stores and transmits information. Simply put, the nodes have agreed on what information is considered correct and what is not and how we store it.

We can draw an analogy with a room - one person shouts out the information he wants the others to keep. Then, everyone in the room works with the information they have stored after the shouting.

For example, a blockchain can store and transmit messages or information about money transfers. Participants in the network verify the information before recording it.

In the room analogy, I shout, “I'm transferring 100 dollars to Sam.” Everyone records that I have 100 dollars less, and Sam has 100 dollars more. If suddenly, before the transfer, I have less than 100 dollars, no one will record the transaction.

Smart contracts

In blockchain, you can create smart contracts in the Solidity language (in the case of blockchain on EVM). A smart contract is a program that runs on the blockchain network. It may contain validation mechanisms, error handling, and other functions.

Again, in the case of the room where we shout commands, a smart contract is me giving each participant in the room the program code in advance: how to react to my commands, what to check, and what to save. And then I shout, “Execute the program with these parameters”. Then, everyone follows the instructions. An example of a simple, smart contract is information storage with the functions of adding and receiving data. The contract code is compiled into byte code and passed to the blockchain participants for execution.

How do we handle the particular requests that will be sent to this smart contract? If we draw an analogy with the backend, it is a service that can process POST and GET requests. POST stores information. GET here sends back the information that we have stored. This is usually how any backend is structured.

During my development on the backend, I got very much used to the arrangement that the API, database, and everything related to data storage and processing happens on my side. And I already, as if behind a wall, provide an interface for the user to work with this data according to a pre-prepared scenario.

For example, user 1 comes and saves content (post, for example) via the POST method. Then, user 2 comes and retrieves this content using the GET method. Users don't know where and how it lies - the backend is a black box for them.

And here we come to one very important part of the blockchain. Let's go back to our examples of nodes or people standing in a room. Let's say, using an analogy with the backend, we have the following every time: I throw an “ADD” method into the blockchain, and then everyone locally calls the method and then can take information from their copy of the blockchain.

So, we have a bunch of different copies in the network from which nodes take information. The trouble with blockchain is that we must pay real money for each write operation. This is paid for by the network currency, which can be bought with real money (or mined, but that's not what we're talking about today).

If we compare blockchain and backend, the picture is as follows:

• In conventional services, we write for free, read for free, and dependently
• In blockchain, we write for a fee and read for free and independently

For example, Telegram has a centralized database. We can always access it for free and download our messages, photos, videos, etc. But if Telegram's servers suddenly go down, we cannot access it.

We have to pay for the EVM virtual machine to execute some smart contract commands, including writing information to the blockchain. It performs some calculations, adds up something, multiplies, multiplies, multiplies, and eventually, a new artifact appears in the blockchain storage, which is updated on all nodes participating in the blockchain.

Any network participant can run a full node with hundreds of gigabytes of blockchain data and work with it locally. You can also use a lightweight version of the node, which will not store the entire blockchain, but you can access the full nodes in the network and retrieve the necessary information through it.

The idea is that each entry in the blockchain is a block that contains a bunch of transactions in which changes in the state of the blockchain occur. Each successive block depends on the previous one in a chain based on hashing algorithms.

In general, it's basic, but it's worth keeping in mind - you have to pay for every sneeze if the data changes. By the way, a contract deployment is also a record in the blockchain and is not cheap!

Deployment of a smart contract

In the backend world, I'm used to roughly the following feature development lifecycle:

• Written the code
• Launched it in Gitlab
• GitLab CI runs tests, checks everything out
• If everything is ok, CI starts deploying a new version of the application to the server

That is, we are used to working this way, and it happens for free. Although conditionally free, because we do pay for the servers. What about blockchain?

In the case of blockchain, we need to write the new code of our “application” (smart contract) into the blockchain. As I wrote above, we have to pay for each record. Before we make a transaction to our smart contract, we need to make a transaction with the placement of the smart contract.

Then, the client/service server will contact any of the nodes to receive or save information in the contract.

A huge number of nodes need to be notified - “guys, here's the bytecode of the contract whose algorithms need to be done on my transactions”. It is imperative to make sure that the same code appears on all the nodes that learn the blockchain, and it will be executed in the same way, regardless of who calls it and regardless of how it is called. The mechanics will be the same and unchanging. Furthermore, there is no way that the smart contract can be somehow altered to work differently on any of the nodes.

Below is an example of a transaction where I deposited a contract into the ETH network quite a long time ago.

It was a test contract that was never used in real. I paid 200 dollars in ETH for its deployment. That is, we hadn't done anything with this contract yet - not a single request, but $200 had already been spent. I'm still sad when I remember this mistaken deployment of the wrong contract...

Data storage

Let's talk about data storage. We are all used to having PostgreSQL, MySQL, MongoDB, Redis, and other services on the backend that allow us to work with data conveniently. In the case of blockchain, there is nothing like that even close.

In blockchain, storage is implemented like variables in a class in other languages. That is, just key values or arrays. There are no relational tables with convenient links, and so on. Just - write to a variable and be happy.

At the moment, I don't know of any other way to organize storage in blockchain. Well, maybe the situation has already changed; maybe when you are reading this, there is such a way - write in the comments.

For example, if we want to store not just in an array? And we want to store information by key - there is mapping for that.

The dollar sign is drawn for a reason - the network commission will be taken for each set.

Aches and pains

In this block, I will discuss things that surprised or angered me. There are many more than are in this document, but I will share the first things that first hit me in my practice.

It's important to note that most of the “pains” are understandable for some logical reason. But that doesn't cancel out the pains for my backend brain.

For example, I'm used to the fact that we can easily go through all elements of anything. It doesn't matter if it's an array or an object or a map. In Solidity, for this purpose, we will have to store separately an array of all keys and then, if necessary, go through all of them and retrieve elements from the map for each key. Well, we also spend gas on additional writing to this array of keys and its initialization.

We can't get all the handy key sorting stuff either.

Logging

The situation with logging is also unpleasant. I'm used to debugging through the debugger in the development environment, but here you should forget even about normal logging.

In Typescript, I'm used to just writing console.log(a) and immediately getting the output in the console. In Solidity, there is console.log, which works only when running in the local hardhat development environment. And, what's great is that after I've split what I need, I have to delete all these logging before the contract deploys because otherwise, the contract weighs more and costs more to deploy, and it won't work at all on the prod.

In the end it turns out that when we run the project already in battle, we want to see what is wrong, we can't see what went wrong. But we can see what went right. There's a system of events within smart contracts. Here's an example: let's say we want to have an event where a new item was added under this index with this value.

We call this event inside the set method, and we can only see the logs when it is successfully executed. If something went wrong, you had multiple calls to contracts, or we had a transaction crash, then the logs are not saved either because the information in the blockchain is rolled back.

Suppose you use a chain of several smart contracts. You have in the first contract called some events, then the second contract is called, which calls other events, and then everything falls that was called inside the second contract. Everything will be deleted exactly completely once and for all.

We have to be very careful when we want to log what is happening inside the blockchain and keep in mind that normal logging, which we are used to, is simply not available to us here.

Another nasty thing is that we can't get information from our transactions in a write function. If we do a transaction that writes something to the blockchain (i.e. a paid transaction), the return will not give anything to our service that integrates with the smart contract. This return only works within the smart contract itself or in view (free) functions.

For example, we would like that when we add a new value to our blockchain, we might want to find out the storage size after saving (screenshot above). That is, we can only find out through events what exactly was added. And to do that, we need to pull events that were triggered within that transaction.

Working with strings

There was a surprise for me here - it is impossible to work with strings normally. The blockchain was not created for strings. Let's move on to the examples.

The code below will work without any problems.

And this code won't work anymore:

I've long been used to working normally with strings, changing characters in strings, cutting strings, concatenating them - all this is not available out of the box. There is also no possibility to display the string length. That is, this code will not compile:

If you really need the length of a string, you can convert it to bytes and then count the number of bytes. But the problem is that some special characters are not converted to bytes 1v1. And some are simply not converted and the transaction may crash.

You could end up writing a smart contract that handles strings and tests on normal strings. Then, a string will arrive that will not be processed, and everything will crash, or the string length will be counted incorrectly because of special characters.

The conclusion on strings is simple: do not work with strings and do not rely on strings inside the contract. If it is important to save strings, then save bytes and rely on bytes, and convert strings to bytes on the service itself.

External call problem

The next complexity, which is an extension of the blockchain's main feature, is isolation. All data that resides on the blockchain is either born inside the blockchain or transmitted to it from the outside. But the blockchain itself can never knock on the outside world - only other smart contracts.

The problem is that all smart contract commands are executed at each participant in the network. And you can't trust an external source, as you can't be sure that the same information will be received on every node. It will happen that each node will have a different version of the blockchain with different data and the blockchain will collapse.

And the trivial task of “getting the current temperature outside” becomes something impossible. While we don't always need the weather, some data (like currency exchange rates or the current state of some external system) is essential. The solution lies in the following approach:

1. We have an Operator Contract, where our service sends a task like “make a request to this server with such and such parameters”.
2. Contract emits Event
3. Separate backend subscribes to this event, extracts information from the event, which says “where and with what parameters to send the request” and the answer “put it here in this contract”.
4. Server sends the request with the right parameters, gets the response
5. Server sends the response to the required contract.
6. What happens next is what should happen with this data.

It turns out to be such a long chain. The sadness of the story is that the money is taken at my first request of the form “go for me on such a request” and at the second request, which is already done by the server that has executed the request.

For example, it takes us 50k gas for each step. We start the transaction, put 50k GAS LIMIT, and think we'll be fine. But, for example, the mechanics of saving the new weather changes - now, when the temperature is above 10 degrees, we need to transfer money to one of the participants. The logic expands, and now it will take, for example, 80k gas per transaction.

In the end, already on the second transaction, the whole chain collapses due to a lack of gas for the transaction. Such a “vegetable garden” around external calls makes such projects more complicated. Most likely, if you have a hard connection to the outside world, you should not choose blockchain for your project.

There is also no normal randomness that cannot be predetermined. This randomness is also provided by different providers “as is” - just a random value is regularly written into the smart contract. But it is dangerous to trust such a thing for real financial projects.

The fact that the value of the block.timestamp variable is set by the block miner deserves special attention. Of course, it is difficult to imagine that the miner will know in advance that it is he who mines the block, and he can substitute the time. Still, there is a hypothetical possibility. This danger is relevant in the context of 15 seconds, and if we rely on minutes and large time intervals, there is no such problem.

Security issues

I do not plan to talk much about security. But I will emphasize an important aspect: everything in the blockchain is visible to everyone. The only thing that is inaccessible to others is your private key. The smart contract code is put out in the open in order to pass audits and so that smart contract users can trust it.

Audit procedure means a company is hired to look at the smart contract code and verify that this particular contract is posted under this address. The issue of the security of the contract is checked, and it does what the developers declare. Next, the auditing company posts information like “this contract has been verified by us - it can be trusted” on its website.

Immutable variables

But even if the contract code is not provided, it can be easily decompiled. For example, the following code has an immutable variable - it is simply replaced everywhere with a constant in the code below.

After deploying this contract and opening it through the decompiler, we see the following:

That is, we get this value of the variable immediately.

Private variables

I'm used to being able to be calm in the backend, and the value of private variables to read without memory access will be problematic. It's the same here - it's just that everyone has access to “memory”.

We called the variable amount private. Deploy the smart contract and then pull its value by a simple code snippet:

You can end up pulling out anything that way. So don't think about storing anything sensitive in the smart contract!

Smart Contract Deployment

It is basically impossible to roll back your changes. A smart contract is deallocated once, and nothing can be changed. It will remain on the blockchain until the end of time and then some.

Upgradeable Smart Contracts

That's why you have to write everything correctly and well at once. I can't do that, so I quickly came up with an interesting workaround - Upgradeable Contracts. Their mechanics work as follows.

1. The first version of the contract (Contract V1) is posted

2. A Proxy Contract is posted and it has the following task: to forward all requests 1v1 to Contract V1 or use its own storage and use only logic from the target contract.

3. Further the user communicates with the proxy contract in the same way as with the main one.

   If it is necessary to update the contract, the admin deploys Contract V2 and, via admin-contract, tells proxy-contract that the implementation is now at the address of Contract V2.

4. Next, the user also communicates with the proxy, and the mechanics from Contract V2 are already executed.

5. Next, the user also communicates with the proxy, and the mechanics of Contract V2 are already executed.

This mechanism has a number of limitations and tricks. For example, variables from the previous version cannot be changed in the new version of the contract. If a variable is no longer needed, it must still be left and depopulated into the new contract.

Of course, this solution and many others already have ready-made workarounds. The main supplier of these developments is OpenZeppelin. So, fortunately, there is no need to reinvent the wheel.