When logic programming meets CQRS

Eve will turn an aspirational design pattern into a reality

Welcome to Part II of my VI-part series about Eve, an exciting and fascinating new programming language.

• I. How Eve unifies your entire programming stack
• II. When logic programming meets CQRS
• III. Throwing off our scope chains
• IV. Smalltalk and protein programming
• V. The rock-solid foundation for Eve’s big vision
• VI. Why Eve will be perfect for realtime apps

Logic programming

Did you know Eve is a logic programming language? Eve is based on Dedalus, which is an extension of Datalog, which is a logic programming language based on Prolog.

The way logic programming works is: you write down a bunch of facts, then you instantly get to query a world in which those facts and all their logical implications are true. If that doesn’t make sense, check out this gentle introduction to Prolog.

Datalog has been around since the 1970s. So if logic programming is so great, why hasn’t it gone mainstream yet? Maybe because it’s less intuitive than procedural and functional programming styles?

No, I think logic programming just hasn’t been fully baked until now. Dedalus’s research paper only came out in 2009, and it looks to me like a major theoretical advance. Dedalus, unlike traditional logic programming, explicitly models updates to facts happening over time as well as over an unreliable asynchronous communication channel. Joseph M. Hellerstein, one of the coauthors of Dedalus, puts it like this:

It may surprise some people that Datalog, the favored language of database theory, had no way to model updates and deletes. By incorporating time into the Dedalus logic, updates and deletes can be captured as deduction of logically timestamped (versioned) data.

And of course, Eve is only coming out now. So we’re just starting to see the potential of logic programming.

CQRS

I think we’ll see that one of the most exciting things logic programming can do is implement the CQRS pattern.

What is it?

Command Query Responsibility Segregation (CQRS) is a gratuitously-long buzz-phrase, but it simply means separating your read-model from your write-model.

For example, in a special case of CQRS called Event Sourcing, you decide that your application’s write-model is a single append-only sequence of event objects like:

• {event: “deposited", accountId: 123, amount: 100}
• {event: “deposited", accountId: 456, amount: 100}
• {event: “withdrew", accountId: 123, amount: 30}

And your application’s read-model looks more like:

• {accountId: 123, balance: 70}
• {accountId: 456, balance: 100}

When your application receives an event, e.g. a deposit, it issues an update to the write-model. When your application needs data, e.g. getting an account’s current balance, it queries the read-model. Meanwhile, you tell your database layer how to always keep the read-model in sync with the write-model.

Why’s it good?

Event Sourcing is already a popular pattern for high-scalability, high-reliability and high-accountability applications. This talk by Greg Young does a great job of explaining its virtues.

Even if you don’t care for Event Sourcing, you probably care about data denormalization. Ok well, any time you think you’re just “denormalizing data”, you’re actually “precomputing a read-model for CQRS”.

Hey, I told you CQRS is a gratuitously-long buzz-phrase for how simple it is. But you can see it’s an important concept to name and care about.

Problem: CQRS is kinda impossible right now

The problem with CQRS is that there’s never been a good way to “tell your database layer how to always keep the read-model in sync with the write-model”. If you zoom into the CQRS diagram above, that arrow labeled “changes” is actually a rickety bridge:

In theory, CQRS is great. But in practice, it’s just not something you can actually do, at least not elegantly and robustly.

The systems that claim to help implement CQRS are usually stream processing systems, e.g.:

• Event Store
• Apache Samza and Kafka-based solutions

You could also use a SQL database to implement CQRS: just put your write-model in one or more tables, then use triggers to react to changes and issue the right INSERT/UPDATE/DELETE operations to sync up your read-model.

Unfortunately, whichever of the above you choose, you’ll have to roll your own messy procedural logic to react to incoming write operations. Meaning, you’ll have to tell the database how to keep two models in sync. You won’t be able to just declare what the relationship between the two models always needs to be.

Also, the consistency guarantees are iffy. When your write-model changes, you can’t guarantee that your entire read-model correspondingly ticks from one consistent snapshot to the next.

The materialized view pattern

Ok, well… what about about using a materialized view pattern to implement CQRS?

In theory, the materialized view pattern is great. But in practice — despite having its own MSDN page — it’s missing a satisfactory implementation! If you zoom into the diagram, that little horizontal arrow is actually a leaky pipe:

Materialized views only work for basic data transformations. They aren’t smart enough to incrementally update themselves in reaction to write-model operations. Plus their consistency guarantees are weak.

I’ve previously pointed out the limitations of materialized views in “Data denormalization is broken”. When I yearned for a “denormalization engine”, I basically meant anything which elegantly and robustly implements the materialized-view pattern.

CQRS seems to me like a key pattern that we should all be using for our data layer — yet it’s currently rare to see a production system with an elegant CQRS architecture, or even hear a mention of “CQRS” in conversation. The sad thing is, I think we don’t bother discussing it because there’s no tool that lets us build it.

Logic programming meets CQRS

Soon, we won’t need the materialized-view pattern, or SQL triggers, or stream processing — because we’ll have Eve.

The latest version of Eve, 0.2, is only an in-memory demo; it’s not nearly database-like enough yet. But I daresay it’s already on track to become the best implementation of CQRS.

In Eve, writing a block with search and bind is kind of like defining a materialized view. Here’s a tiny one-record materialized view / CQRS thingy:

I bound c to always equal a + b. Right now a is 5, b is 6, and c is 11. If I increment b to 7, then I’ll automatically see a value of 12 for c.

Look, this is CQRS! My write-model is a and b, and my read-model is c. It’s a trivial example, but it illustrates the core concept. I’ll show a more elaborate example of bind in Part VI.

With Eve’s bind operation, you can sync your read-model with your write-model, making the CQRS pattern a reality. Look at that “change” arrow now:

Now that we’ve seen that logic programming is a promising approach to making CQRS a reality, I should also mention Datomic, which is perhaps the most well-known logic programming database in the market today. This post and this talk showcase CQRS architectures built on Datomic. Neither of them is “pure”; they both require a custom mix of other systems to do some of the work. Still, they’re making promising progress toward CQRS independent from Eve.

**Next post:**III. Throwing off our scope chains