SQLite the New Hotness?! 🤔

SQLite is popping up in many places all of a sudden as hot new tech. While it's nowhere near as hot as AI/ML right now, it is amazing that such an old technology is seeing a renaissance.

SQLite is a file-based database that was initially released 23 years ago. I often tell other developers it is the fastest database, but it just doesn't scale off of one server. However, because it is a file, it's extremely easy to use and also includes many advanced features like full-text search and GIS. Because of its ease of use, it is used in many places behind the scenes, such as web browsers, operating systems, embedded applications, and more.

So why is SQLite all the rave right now? Below, I'll outline some of the newest SQLite tech with a summary of why you might want to use it.

Blockchain Powered Distributed Data

SQLite has a few distributed implementations that the Bitcoin Bros would be proud of. The Raft protocol and Paxos protocol are consensus protocols that can be used in blockchain-like use cases. In this case, they were implemented with SQLite to create a distributed database. This helps get SQLite off of one machine and onto too many. Because of its distributed nature, the trade-off is that you have slower write performance. However, you gain reliability and availability. So, if you want to distribute your database to multiple machines, try out one of the following:

• rqlite
• SQLite Cloud
• BedrockDB

Usage

RQLite, SQLite, and BedrockDB are not a drop-in replacement for SQLite in your application code. RQLite has an HTTP API which you can use and all have client libraries. The client libraries may be a drop-in replacement for SQLite in your application code, but otherwise, you will probably have to do a small lift to convert over to the appropriate client library. Additionally, Bedrock emulates MySQL. So if you can use MySQL in your application, then you can also drop it in BedrockDB.

Primarily More Fun with Distributed Data

The second class of the distributed SQLite databases I've found on a high level distributes read replicas and passes write operations to a "primary" server without the use of blockchain algorithms. These implementations get you off of one machine with better write performance, in theory than the Raft protocol implementations. Two popular implementations I found are:

• LiteFS
• Turso

Usage

LiteFS as the name applies works more like a file system and is more a drop in replacement for your current SQLite application code. You will need to set up the replication mechanism, but you will not need to change your application code. Turso requires the usage of libSQL, which, while compatible with SQLite, will require the usage of a new client library in your application code. So, you will probably need some small tweaks to your application to get it working. Turso also offers an HTTP API.

Durable SQLite

There are several solutions that allow you to replicate your SQLite database. While these solutions don't really get you off of one server unless you can configure your application for write operations to one server, they do allow you to persist your database and make it available for read replicas. You'll want to use this method if you are trying to permanently preserve your data off your server or you only need one server that does the write operations and many more that do reading. This may sound similar to the previous implementations but differs because write operations are not transparently handled. It is up to you to solve this. This could be useful in storing structured data for a pipeline or for small applications that only need one server with a way to back up your database.

There are several options for this type of solution:

• LiteStream
• Verneuil
• LiteReplica

Usage

The good thing about this type of solution is that they are often drop-in replacements for your application. You will need to set up the replication mechanism for your application, but your application code will remain the same.

Yes, There's More!

Yes, there are more SQLite implementations, but I found the ones I listed above to be more complete or easier to use right now. So keep an eye out on the SQLite space because it is moving fast.