A Step by Step Guide to immudb — the open source immutable database

May 18th 2020 1,876 reads

@dennis-zimmer

Dennis Zimmer

Co-Founder & CTO, CodeNotary

immudb is lightweight, high-speed immutable database for systems and applications.

With immudb you can track changes in sensitive data in your transactional databases and then record those changes indelibly in a the tamperproof immudb database.

This allows you to keep an indelible history of, say, your debit/credit transactions.

immudb is open source under the Apache v2.0 License, and can be found here (there is also a more comprehensive documentation):

https://github.com/codenotary/immudb

Traditional transaction logs are hard to scale, and are not immutable. So there is no way to know for sure if your data has been compromised.

You can find an example video here:

https://www.youtube.com/watch?v=g7msV9NcwNA&t=1s

How it works

As such immudb provides unparalleled insights retro-actively, of what happened to your sensitive data, even if your perimiter was compromised. immudb provides the guarantatee of immutability by using internally a Merkle tree structure.

immudb gives you the same cyrptographic verification of the integrity of data written with SHA-256 like classic blockchain without the cost and complexity associated with blockchains today.

immudb has 4 main benefits:

• immudb is immutable. You can only add records, but never change or delete records.
• data stored in immudb is cryptographically coherent and verifiable, like blockchains, just without all the complexity and at high speed.
• Anyone can get started with immudb in minutes. Wether in node.js, Java, Python, Golang, .Net, or any other language. It’s very easy to use and you can have your immutable database running in just a few minutes.
• Finally, immudb is Open Source. You can run it on-premises, or in the cloud and it’s completely free. immudb is governed by the Apache 2.0 License.

immudb runs on Linux, FreeBSD, Windows, and MacOS, among other systems derived from them, such as Kubernetes and Docker.

Getting started

You can either build Docker images based on the Dockerfiles in the GitHub repository for the most common architectures or use the prebuild ones on Dockerhub for Linux.

immudb Dockerhub

docker run -it -d -p 3322:3322 -p 9497:9497 — name immudb codenotary/immudb:latest

standalone Binaries

If you want to build the binaries yourself, simply clone this repo and run one of the following commands based on your operating system.

# Linux
GOOS=linux GOARCH=amd64 make immudb-static
# macOS
GOOS=darwin GOARCH=amd64 makeimmudb-static
# Microsoft Windows
GOOS=windows GOARCH=amd64 make immudb-static

Then you can run immudb the immudb server

# run immudb in the foreground 
./immudb 
# run immudb in the background 
./immudb -d

install immudb as a service

# install immudb service 
./immudb service immudb install 
# check current immudb service status 
./immudb service immudb status 
# stop immudb service 
./immudb service immudb stop 
# start immudb service 
./immudb service immudb start

The immud linux service is using the following defaults:

• user: immu
• group: immu
• configuration: /etc/immudb
• data: /var/lib/immudb
• logs: /var/log/immudb
• Service Port: 3322 (immudb)
• Prometheus Exporter Port: 9497

Performance

As immudb is often compared to Amazon QLDB, we did a performance benchmark using a simple demo application to write data (without using any unfair optimization).

Test setup:

• 4 CPU cores
• Intel(R) Xeon(R) CPU E3–1275 v6 @ 3.80GHz
• 64 GB memory
• SSD

Prometheus and Grafana monitoring

immudb has a built-in prometheus exporter that publishes all metrics at port 9497 (:9497/metrics) by default. When running a Prometheus instance, you can configure the target like in this example:

- job_name: 'immudbmetrics'
    scrape_interval: 60s
    static_configs:
         - targets: ['my-immudb-server:9497']

There is a Grafana dashboard available as well: https://grafana.com/grafana/dashboards/12026

Common Use Cases

We already learned about the following use cases from users:

• use immudb to immutably store every update to sensitive database fields (credit card or bank account data) of an existing application database
• store CI/CD recipes in immudb to protect build and deployment pipelines
• store public certificates in immudb
• use immudb as an additional hash storage for digital objects checksums
• store log streams (i. e. audit logs) tamperproof

No programmer?

Actually in case you’re not a programmer but still want to use immudb just to play around or within scripts, you can use immuclient.

# Linux
GOOS=linux GOARCH=amd64 make immuclient-static 
# Microsoft Windows
GOOS=windows GOARCH=amd64 make immuclient-static

In case you have no idea how to build it, you can use the following Docker command and procedurel:

git clone 

https://github.com/codenotary/immudb.git

# Linux
docker run -it --rm -v $(pwd):/src golang:1.13-stretch sh -c 'cd /src && GOOS=linux GOARCH=amd64 make immuclient-static'
# Microsoft Windows
docker run -it --rm -v $(pwd):/src golang:1.13-stretch sh -c 'cd /src && GOOS=windows GOARCH=amd64 make immuclient-static'

Now you’ll find the immuclient binary in the repository folder and ready to be used. 

./immuclient --help

Add a record to immudb

# same system where immudb server is running
./immuclient safeset mykey myvalue
# immudb server runs on a remote system
./immuclient -a <immudb-ip> safeset mykey myvalue

You'll receive something similar to:

./immuclient safeset k1 v1
index: 307
key: k1
value: v1
hash: 4a6a18172eba5a3ea49a3caf147ac405c874ed4c922cc7dafe0dce5ff85f35aa
time: 2020–05–13 04:01:30 -0400 EDT
verified: true

Get the record from immudb

# same system where immudb server is running
./immuclient safeget mykey
# get the value history
./immuclient history mykey
# immudb server runs on a remote system
./immuclient -a <immudb-ip> safeget mykey

The

safeGet 

safeSet 

Now you could store any kind of data, like the content of a sensitive database field, public certificate or a even a configuration file.

Let’s try with a local Dockerfile and make sure there are not new lines or special characters in our value.

./immuclient safeset Dockerfile1 $(echo -n "$(cat Dockerfile)" | base64 -w 0)

To get the data back, you need to make sure to convert it again.

As the output of safeget contains more than just the value, as seen here:

./immuclient safeget Dockerfile1
index:          309
key:            Dockerfile1
value:          RlJPTSBnb2xhbmc6MS4xMy1zdHJldGNoIGFzIGJ1aWxkCldPUktESVIgL3NyYwpDT1BZIC4gLgpSVU4gR09PUz1saW51eCBHT0FSQ0g9YW1kNjQgbWFrZSBpbW11YWRtaW4tc3RhdGljCkZST00gdWJ1bnR1OjE4LjA0Ck1BSU5UQUlORVIgdkNoYWluLCBJbmMuICA8aW5mb0B2Y2hhaW4udXM+CgpDT1BZIC0tZnJvbT1idWlsZCAvc3JjL2ltbXVhZG1pbiAvdXNyL2xvY2FsL2Jpbi9pbW11YWRtaW4KCkFSRyBJTU1VX1VJRD0iMzMyMiIKQVJHIElNTVVfR0lEPSIzMzIyIgoKRU5WIElNTVVBRE1JTl9JTU1VREItQUREUkVTUz0iMTI3LjAuMC4xIiBcCiAgICBJTU1VQURNSU5fSU1NVURCLVBPUlQ9IjMzMjIiIFwKICAgIElNTVVEQl9NVExTPSJmYWxzZSIgCgpSVU4gYWRkZ3JvdXAgLS1zeXN0ZW0gLS1naWQgJElNTVVfR0lEIGltbXUgJiYgXAogICAgYWRkdXNlciAtLXN5c3RlbSAtLXVpZCAkSU1NVV9VSUQgLS1uby1jcmVhdGUtaG9tZSAtLWluZ3JvdXAgaW1tdSBpbW11ICYmIFwKICAgIGNobW9kICt4IC91c3IvbG9jYWwvYmluL2ltbXVhZG1pbgoKVVNFUiBpbW11CkVOVFJZUE9JTlQgWyIvdXNyL2xvY2FsL2Jpbi9pbW11YWRtaW4iXQ==
hash:           dfca217e2d87dccb8fd3fe8c1b49e620cc4ece8dc9c9fc2384cb6f6c9617eddb
time:           2020-05-13 05:19:19 -0400 EDT
verified:       true

the command is a bit more complex

./immuclient safeget Dockerfile1 | grep "^value" | cut -d":" -f2 | xargs echo -n | base64 -di

immudb SDKs

There are many options for developers using the SDKs for Go Java, Node.js, Python

These can be found in a developer jumpstart guide:

https://docs.immudb.io/master/jumpstart.html#contents and the API descriptions:

https://docs.immudb.io/master/sdks-api.html

There is also a great Go SDK video tutorial: https://www.youtube.com/watch?v=qCC_AghFiw4

Check out immudb, the immutable database, written in Go: https://github.com/codenotary/immudb