128 reads

When ORMs Disobey: A Mongoose Story

by Ramit MittalFebruary 28th, 2021

Too Long; Didn't Read

In hindsight, it seems like mongoose was doing the sensible thing by simply ensuring that the document existed in the database.

Company Mentioned

featured image - When ORMs Disobey: A Mongoose Story

Note: All examples are very contrived; minor suspension of disbelief is required.

A while ago, my team was experimenting with some

pre-find

hooks in mongoose that modified the query conditions before executing it. We wanted to apply a filter at a global level, for example, only return records that have

age >= 18

from the

persons

collection.

Baking this logic inside the call itself seemed like a good idea initially. It saved us from having to modify every existing

Model.find()

call in the codebase. This also provided a safeguard against devs forgetting to add the condition to every

Model.find()

call in the future.

"use strict";

const mongoose = require("mongoose");

const personSchema = mongoose.Schema({
  name: String,
  age: Number,
});

function preFindHook(next) {
  if (this._conditions.age >= 18) next();
  else next(new Error("Invalid query"));
}

personSchema.pre("find", preFindHook);
personSchema.pre("findOne", preFindHook);

const PersonModel = mongoose.model("Person", personSchema);
module.exports = PersonModel;

It worked perfectly; until a bunch of

PATCH

endpoints started to fail randomly.

router.patch('/persons/${personId}', async (req, res, next) => {
  const { personId } = req.params;
  const updatedPersonInfo = req.body;

  const existingPersonRecord = await PersonModel.findOne({
    _id: personId,
    age: { $gte: 18 },
  });

  if (updatedPersonInfo.name) {
    existingPersonRecord.name = updatedPersonInfo.name;
  }

  await existingPersonRecord.save();
  res.json({ person: existingPersonRecord.toObject() });
})

We started searching for the issue using

console.log

statements in route controllers. It seemed as if our recently registered

pre-find

hook was being triggered on

Model.save()

as well.

router.patch('/persons/${personId}', async (req, res, next) => {
  ...
     existingPersonRecord.name = updatedPersonInfo.name;
   }
 
+  console.log('Reached this far without error');
   await existingPersonRecord.save();
   res.json({ person: existingPersonRecord.toObject() });
 })

Reached this far without error
Error: Invalid query
    at model.Query.preFindHook
    at callMiddlewareFunction

To confirm this, we turned on debug mode in mongoose

mongoose.set('debug', true)

and removed the

pre-find

hook.

Mongoose: people.findOne({ _id: ObjectId("6031d9f91973241ccb4848e9"), age: { '$gte': 18 } }, { projection: {} })
Reached this far without error
Mongoose: people.findOne({ _id: ObjectId("6031d9f91973241ccb4848e9") }, { session: undefined, projection: { _id:

In debug mode, mongoose logs every query that it runs. It is clear that

findOne()

was executed here when

save()

was called. This

findOne()

call did not specify an age property in the filter. This made the

pre-find

hook throw an error.

The very idea of mongoose executing a different operation than the one it was asked to was baffling to me. I spent the next hour scouring the web for hints or explanations, finding none.

The mystery unraveled when I read the source code of

Model.save

as a last resort. The important parts are shown below. Complete source code can be found here.


Model.prototype.$__handleSave = function (options, callback) {
  if (this.isNew) {
    this[modelCollectionSymbol].insertOne(
      obj,
      saveOptions,
      function (err, ret) {
        if (err) {
          _setIsNew(_this, true);

          callback(err, null);
          return;
        }

        callback(null, ret);
      }
    );
  } else {
    const delta = this.$__delta();

    if (delta) {
      this[modelCollectionSymbol].updateOne(
        where,
        delta[1],
        saveOptions,
        (err, ret) => {
          if (err) {
            this.$__undoReset();

            callback(err);
            return;
          }
          ret.$where = where;
          callback(null, ret);
        }
      );
    } else {
      const optionsWithCustomValues = Object.assign({}, options, saveOptions);
      this.constructor
        .exists(this.$__where(), optionsWithCustomValues)
        .then((documentExists) => {
          if (!documentExists) {
            throw new DocumentNotFoundError(
              this.$__where(),
              this.constructor.modelName
            );
          }

          callback();
        })
        .catch(callback);
      return;
    }
  }
};

Mongoose internally tracks whether a document is ‘new’ or not. It also tracks the fields (if any) that have been updated on the document. This information is used to decide what operation is performed when the

.save()

method is called on the document.

When the document is ‘new’, mongoose performs an insert.
When the document is not new, mongoose calculates the ‘delta’.
If any fields have been modified, mongoose performs an update.
If no fields have been modified, mongoose checks that the document exists in the database using findOne.

What We Could Have Done Better

The reason why only the PATCH calls failed was that in a PATCH operation, which is a partial update, there is a possibility that none of the fields are updated. In this case, we should not have called

.save()

on the document in the first place.

The hook we wrote was flawed as well. The

_conditions

is a private property of 3rd party code. We should not be messing with it even if JavaScript allows us to.

In hindsight, it seems like mongoose was doing the sensible thing by simply ensuring that the document existed in the database.

Mongoose is a (heavy) abstraction layer over Mongo. It is incorrect to think of its operations as simple wrappers to those of Mongo’s native driver.