Hypothesis: CQRS Models with Blockchain as the Persistence Layer Will Work

The article shows how to apply the CQRS pattern with Blockchain technology to solve a business case that it’s not possible using the native features that the Blockchain platform offers.

You will find in this article:

• Problem context: What we are aiming to solve. From business side and technical side
• Example scenario: Everything seems easier if we have an example
• Which Blockchains types can take advantage of this approach
• Introducing CQRS concepts
• Explaining why CQRS and Blockchain
• Explaining why Event-Sourcing and Blockchain maybe not
• Source code example to solve the example scenario
• Recap of what has been explained
• List of compatible blockchains with this approach

I will change the article when I find things that are wrong!!

📢 ⚠️ The article might be ignoring some Blockchain platforms that might solve better the problem-solution described in this article because I might not be aware of. In that case, I will appreciate a comment to update the article with that new information. The Blockchain industry changes fast and being up to date is nearly impossible without full dedication. Thanks for your consideration.

ℹ️> You might want to skip the next part which is more the context of the solution, if you want to read the technical approach jump to “The Case” section

Why do we need CQRS Models? What we have today

In order to know why we might need CQRS models on Blockchain, let me explain the problem we want to solve first.

All products are the specific solution implementation over a problem space. The solution space has to be guided via the problem space, otherwise we might be building something that it’s not needed by the user.

Problem-Solution space diagram

The problem space is your domain. A domain is what the business wants to solve. We might have the case of an asset management platform. The domain is the asset management.

We might want to divide the domain into sub-domain to have a better way to organize our solution space, a basic Divide and Conquer strategy. We might want to define/implement each differently on depending on the requirements. It might be more familiar to you if you know Domain-Driven Design or Microservice Architecture, but it’s not required to follow this article.

We want to define those sub-domains because, one reason but not the only one and nor the most important, it will just couple one sub-domain with Blockchain. So, we contain all possible problems with Blockchain on that specific sub-domain and not let all the application suffer of that undesired coupling.

You might want to use Blockchain on more than one sub-domain, it might be needed, but I would suggest you to not start that big since you might need first get experience with Blockchain and, if at the end you need to drop that implementation, you will have to switch that sub-domain solution to another. So, controlling the accidental complexity that Blockchain brings to the system into just one sub-domain will be enough work and it will give you enough headache, don’t start with two sub-domains using Blockchain. Just a recommendation. 🙂

On each sub-domain, you will have different classes that represents a business concepts.

Example of Entities that might belong to a sub-domain

That business concept, we will refer it as a Domain model. Domain model is a conceptual model of a domain that contains both data and behaviour. The Domain model contains the code to solve the business problem. So, it gives value to the platform since it solves what the user need. As a developer, and as a company, you want to invest your resources on those models because, usually, it’s from where you get money.

We can agree that a Domain Model is always incomplete and imperfect, and some times useful. So, we need to iterate fast over our assumptions and adapt the business needs to the real user needs, that they might change faster than the company solution.

In order to know that we are building the correct tool, we want to have short delivery times and a fast feedback loop. The faster is this feedback loop, the more competitive your product is in the market. In essence, be agile.

Image found at electric-cloud.com

In the few cases, the solution is to use Blockchain technology. Which are that those cases? It’s out of scope for this article. 🙃

The case

Let’s assume the solution space requires an immutable ledger to interact with 3rd party actors over a shared & trusted network. An actor in the system is anyone that interacts with the system we have defined.

Our case scenario is a company that provides an Asset Management System. As a development team, we have seen that Blockchain seems to be a promising technology that fits our initial needs. So, we do a research of which Blockchains we could use.

Third option of not use Blockchain technology has been dismissed in this article, but you should consider it :-)

Some Blockchains have a better approach than others to solving any business problems. Let’s group two main ways to solve it, Smart Contract and Predefined features/contracts.

Smart contract approach, Ethereum style. It helps represent the whole business model. Important info for the reader, I’m not an Ethereum expert ⚠️.

Predefined features approach, NEM style. It provides a set of predefined transactions that combined brings a higher way to interact. Problem, the domain model cannot be defined to match our business like we might do with traditional tools. Important info for the reader, I’m more familiar with NEM Blockchain 🙂.

Properties that we like to have

1. Short development cycles
2. Business Logic upgradable easily and at any moment
3. Rich domain model
4. Immutable data store
5. Being used by anyone in the network

There is different ways to implement a Domain Model with Blockchain. In my humble opinion, all methods I know are too expensive and the development time too high compared to “traditional” technologies. Be sure Blockchain cost overhead is worth.

As development team, we have chosen the NEM Blockchain due to its focus on Assets.

So**, which options** we have to build a useful Domain Model? Because the first business requirement is use tokens, we find that NEM provides this model in its core. So, we have already what we need using NEM Blockchain. Path of Domain Model to Fits the current Blockchain Model, we have been lucky :-)

The article focuses on the path to blue.

ℹ️> In case of NEM2, aka Catapult, you will have the option to extend the Blockchain behaviour for private chains, but I want to share the case where you haven’t this option to help projects using other Blockchain platforms that hasn’t the option to extend the on-chain behaviour but they provide a transaction that can contain a raw text message.

The problem for the development team

There is a business shift! The business team has seen that the customers need non-fungible assets. An asset is non-fungible when it is unique, has an unique owner.

• Fungible Asset: Tokens, there’s 100.000 X tokens, they are all equal between them.
• Non-Fungible Asset: A Car, you own the Car with ID 1202.

The business needs to put the product fast to the marked to verify the business assumption of value proposition. That’s why the Product Owner shared the concern that they need to try things fast and pivot faster with the development team.

After the first deployment, the development team realizes the NEM Blockchain transactions doesn’t represent all things that the business needs, for example, it hasn’t the native support for Non-Fungible Assets.

As development team, we have two options:

1. Rewrite everything to use a Blockchain with Smart Contract support. It implies learning a new technology again, its limits and re-adapt what we have with the new technology. If we are lucky, we just have few things, but if had 2–3 components (mobile app, web app, server), re-write some of that logic might be painful and a resource waste. ❌
2. Adapt the way we interact with the Blockchain to have more freedom defining Domain Models. The development team has acquired experience with the Blockchain and they know the limits of that technology plus they had started estimating better the features related to Blockchain, we don’t want to start this process again. Plus we can reuse part of that we already have. ✅

📢 We choose the 2nd option because otherwise this article doesn’t make any sense. By when you finish the article, if you still think I should explain why you shouldn’t migrate to a Smart Contract Blockchain, just leave a comment and I will reply you! 😁 Just take into account that there’s no perfect solution, just some useful.

Let’s proceed with the solution

The development team has to model a non-fungible asset on top of a blockchain that does not offers it natively, it looks for alternatives.

The dev team does a research of different design approaches and finds CQRS pattern. CQRS stands for Command Query Responsibility Segregation. They read the article because it’s worth.

In case you haven’t time to read that full CQRS article, here a brief, incomplete and probably poor introduction but still useful to follow the article.

Using the CQRS pattern, the development team could create a virtual representation of a non-fungible assets using NEM native features, like transactions messages.

Introducing CQRS concepts

Command Query Responsibility Segregation pattern says that you can use a different model to update information (Command) than the model you use to read information (Query). (source)

Image inspired in CQRS article

So, we will use Commands Models to update the state and Query Models to fetch the state.

CQRS is often used with Event-Sourcing, but it’s not necessary to be used together. Event-Sourcing ensures that every change in the state is captured in an event object and persisted.

Event-Sourcing has some desired benefits that we want to apply on Blockchain.

• Complete Rebuild: We can rebuild the application state re-running the events from the event log on an empty application.
• Temporal Query: We can determine which were the application state at given point.
• Event replay: If some past event was incorrect, we can compute the consequences and apply a fix.

With Blockchain technology, some points are more difficult than others.

What’s an Event Log? It’s a database where all events are persisted.

A Command and an Event doesn’t contain logic, just data. They are Data Transfer Objects (DTO).

A Command shows the intention of changing the state. An Event shares that something happened. There’s a huge difference. A Command might fail, meanwhile an Event reflects something that already happened in the system.

Why makes sense to use CQRS Models with Blockchain?

Recap from where we come, we had the need to represent a business thing that should change its state over time and still use Blockchain.

A CQRS Model helps us to define a set of Commands that we will store on the Blockchain and publish our Query Models with a public API.

Storing the Commands on-chain helps us to re-create the state and audit the state cache that a third party is providing to know it’s valid or invalid.

Sharing the commands help us to quickly let others know how to interact with our Domain Model though a trusted network without the need of sharing our Domain Model internal logic.

As a team, we can choose the level of shared information that we want.

Case — Sharing Commands: people will need to a) rely on our API to read b) emulate our model logic in case something is wrong and prove we are cheating

Case — Sharing Commands & Model Logic: people will a) rely on our API to read or b) create an application to execute our model logic and don’t rely to our API.

Case — Sharing all the application: people will a) rely on our API to read b) start all the application on their end and don’t use us.

There’s a grey limit about sharing the Model Logic and everything.

In our case, we opt to share the Commands & Model Logic. We want others to be able to execute this by themselves if they want, and rely to us for some specific logic, like complex Query Models.

With CQRS model, we are able to scale differently the Command Model and the Query Model. We want the system to respond to complex query fast, but we don’t mind to have slow write since the asset creation or management isn’t as frequent as reading the data.

Also, this approach allow us to choose just share the Commands if we would like to. It’s very important for a company choosing what to share and what not to share.

The imaginary company differentiation is how to manage the information, not controlling the asset creation.

We will use a NEM Account to store all commands.

👩‍💻 Code!! 👨‍💻

We take a look into the specification. It says:

<a href="https://medium.com/media/5d9dea9d0868ff00f58d8b3e007045db/href">https://medium.com/media/5d9dea9d0868ff00f58d8b3e007045db/href</a>

A possible UML diagram could be

The implementation is different due it evolves faster

️⚠️ The following code is extracted from the nem2-nonfungible-asset library implementation. It might have changed by the time you read the article. I suggest you to check the files referenced below to have a more accurate code example.

How a Command looks like:

<a href="https://medium.com/media/ab51383513cbaf75dfa21f5c9f0ee1f3/href">https://medium.com/media/ab51383513cbaf75dfa21f5c9f0ee1f3/href</a>

It does 3 things:

• Be created with the required information (lines 5–7)
• Create a specific Transaction of NEM2 with a specific text message (lines 36–41)
• Recreate from a Transaction (lines 9–23)

Notice that some information is missing until it’s pushed into the network, like the signer (owner).

In the future, we might like to store the transaction info inside the Command to know that they are executed in order.

The Domain Model, Asset, is designed as:

<a href="https://medium.com/media/6e06592ce0be178ea64876e27c7784ac/href">https://medium.com/media/6e06592ce0be178ea64876e27c7784ac/href</a>

We have a way to re-create the state given all commands (lines 12–23). They delegate into the apply method where, because of TypeScript limitations, maps the commands into the methods that have the logic of apply the change.

On each apply method, we will place the business logic. In case it fails, we should have different strategies to report the failure to the creator. I will explain this in another post.

How to read from the Blockchain

<a href="https://medium.com/media/23e297ab2821e8406ba3efcd152f768b/href">https://medium.com/media/23e297ab2821e8406ba3efcd152f768b/href</a>

We fetch all Transfer Transactions (line 20) and we get each Command (lines 21–27). After that, we re-construct the state of the Asset.

The Repository has some limitations due to how the Catapult is implemented, but a feature request is already in progress (#7 & #8).

The most critical is being unable to sort by transaction height, we want to read from older transaction to newer. It’s still not implemented on the current Asset Repository.

We might need to fetch all transactions in memory and then reverse the order meanwhile those features aren’t implemented.

Pending things to be implemented

• State cache persisted
• Query models on that state cache
• Incremental command application via server side notifications (WebSockets)

Drawbacks

• What if a command is included before another announced previously?

We assume that it will be concurrency on the writing and they might cause undesired command application. In case of the Domain Model has dependency on the order, it might cause some race-condition problems, but it might not be the case. We are aware of that it might happen and we develop our Domain Model taking that into account.

• The writing is as slow as the Blockchain Block

Yes, it’s. When we had chose a Blockchain/DLT technology, we were aware of the consequences. We cannot make it go more faster on writing, but with CQRS, we can scale how we read. The use case should have this properties, more reads than writes.

Recap

We tried a first iteration of a CQRS Model with Blockchain. It seems to be viable to apply and extend the features.

The solution proposed isn’t fully implemented and there’s still room for improvement.

Give it a try on your next library and share your experience with us and others!

<a href="https://medium.com/media/a511fb273fbe6d94dbd501d00490d73c/href">https://medium.com/media/a511fb273fbe6d94dbd501d00490d73c/href</a>

You can check how the library evolves here.

List of Blockchains/DLT that might benefit from this approach

• NEM2, aka Catapult. Library example. https://github.com/aleixmorgadas/nem2-nonfungible-asset
• NEM1. No library example yet

📢 Add a comment with the Blockchain or Distributed Ledger that you think it could benefit from this approach and it’s not listed yet. If you think you could write the same library for another Blockchain, feel free to and share your lib here, so others could find how to implement CQRS models on more than one Blockchain.

References

• CQRS: CQRS stands for Command Query Responsibility Segregation. At its heart is the notion that you can use a different model to update information than the model you use to read information.
• NEM2 Blockchain: NEM is a blockchain platform built from the ground up to be simple for developers to use. NEM gives you direct access to a specialized set of tested and secure on-blockchain features using your favourite programming language.
• nem2-nonfungible-asset library: The goal of the library is manage non-fungible assets that on NEM2, aka Catapult, in a deterministic way. It uses the concepts shared in this article.