Don’t Just Relax; Slouch: A JS Client for CouchDB that Does the Heavy Lifting

With the recent release of CouchDB 2, the CouchDB ecosystem is on fire! There is now an out-of-the-box way of setting up a multi-master cluster of nodes that can scale to handle a ton of data. And, tools like PouchDB can allow your app to talk directly to CouchDB to enable your apps to be . It’s an exciting time to be using CouchDB at the data layer of your app! offline-first Even if you focus on a frontend-heavy design, chances are, you’re going to have to do a lot of coding on your backend to communicate with your CouchDB instance, including subscribing to real-time changes. When we first started working on a digital dropbox and grading system, we relied heavily on the awesome client. Through a lot of trial-and-error though, we found that a lot of messy logic, mostly to handle large data sets and transient database errors, was starting to clutter the application layer. We were also left wanting something flatter than nano that more closely mimicked the raw CouchDB RESTful API. We didn’t want to have to bind the client to a specific database, as in practice, this leads to a lot of overhead. We also wanted promises to be a native part of our client as we consider them essential to keeping our code manageable. Quizster, nano So, was born. Slouch Slouch provides all the that you’d expect to find in a CouchDB client and a lot more. Let’s take a look at some of the cool stuff. typical features Fault Tolerance Slouch attempts to retry your requests in the event that the connection to your database is temporarily lost. Moreover, it retries through max_dbs_open errors and other transient errors to prevent your application from crashing every time there is a hiccup on the database. StreamIterators These days, working with large data sets is practically a requirement. Sure, you can paginate your data, but CouchDB supports streams and StreamIterators make working with these streams a breeze. StreamIterators are available for all requests that can return a large number of docs, i.e. , , , db.changes() db.view() doc.all() system.updates() StreamIterators work with promises and reduce your code by introducing a simple and powerful pattern. Example 1 — Sequentially process all docs in a DB slouch.doc.all('mydb', { include_docs: true }).each(function (item) { // If we return a promise then the all() iterator won't move on // to the next item until the promise resolves. This allows us // to iterate through a large number of docs without consuming a // lot of memory. It also allows us to control the flow of the // docs and process them sequentially so that we don't end up // thrashing the processor with too many concurrent promises. return Promise.resolve('foo => ' + item.foo); }).then(function () { // Done iterating through all docs }).catch (function (err) { // An error occurred }); Example 2 — Process a max of 5 docs concurrently var Throttler = require('squadron').Throttler;var throttler = new Throttler(5); slouch.doc.all('mydb', { include_docs: true }).each(function (item) { return Promise.resolve('foo => ' + item.foo); }, throttler).then(function () { // Done iterating through all docs }); Helper functions The CouchDB conflict policy is a very powerful convention and is at the heart of CouchDB and its offline capabilities. Sometimes however, conflicts don’t matter to you and it’s nice to have functions to ensure that your logic persists through these conflicts. upsert Sometimes you just want to force a creation or update even if there is a conflict: slouch.doc.upsert('mydb', { _id: '1', foo: 'bar' }); getMergeUpsert getMergeUpsert allows you to make partial updates without regard to conflicts: // Create a docslouch.doc.create('mydb', { _id: '1', foo: 'bar' }).then(function (doc) { // Add the \`yar\` attr to the doc and ignore any conflicts return slouch.doc.getMergeUpsert('mydb', { \_id: '1', yar: 'nar' }); }); Slouch and Beyond This just scratches the surface of what you can do with . We’ve found to work very well in production for and using it has greatly simplified our application layer as a lot of the boilerplate code is now at the layer. Slouch Slouch Quizster Slouch Our next step is to use to open source our layer so that it is a little easier to implement scaling replication and change listening for CouchDB. Slouch Spiegel About the Author Geoff Cox is the Co-Founder of a digital dropbox and grading system. uses a full stack of JS and runs CouchDB and PouchDB at the data layer. Quizster, Quizster,