Hackernoon logoWhen ORMs Disobey: A Mongoose Story by@ramit

When ORMs Disobey: A Mongoose Story

Ramit Mittal Hacker Noon profile picture

@ramitRamit Mittal

Software Dev @ Attosol

Note: All examples are very contrived; minor suspension of disbelief is required.

A while ago, my team was experimenting with some

pre-find
hooks in mongoose that modified the query conditions before executing it. We wanted to apply a filter at a global level, for example, only return records that have
age >= 18
from the
persons
collection.

Baking this logic inside the call itself seemed like a good idea initially. It saved us from having to modify every existing

Model.find()
call in the codebase. This also provided a safeguard against devs forgetting to add the condition to every
Model.find()
call in the future.

"use strict";

const mongoose = require("mongoose");

const personSchema = mongoose.Schema({
  name: String,
  age: Number,
});

function preFindHook(next) {
  if (this._conditions.age >= 18) next();
  else next(new Error("Invalid query"));
}

personSchema.pre("find", preFindHook);
personSchema.pre("findOne", preFindHook);

const PersonModel = mongoose.model("Person", personSchema);
module.exports = PersonModel;

It worked perfectly; until a bunch of

PATCH
endpoints started to fail randomly.

router.patch('/persons/${personId}', async (req, res, next) => {
  const { personId } = req.params;
  const updatedPersonInfo = req.body;

  const existingPersonRecord = await PersonModel.findOne({
    _id: personId,
    age: { $gte: 18 },
  });

  if (updatedPersonInfo.name) {
    existingPersonRecord.name = updatedPersonInfo.name;
  }

  await existingPersonRecord.save();
  res.json({ person: existingPersonRecord.toObject() });
})

We started searching for the issue using

console.log
statements in route controllers. It seemed as if our recently registered
pre-find
hook was being triggered on
Model.save()
as well.

router.patch('/persons/${personId}', async (req, res, next) => {
  ...
     existingPersonRecord.name = updatedPersonInfo.name;
   }
 
+  console.log('Reached this far without error');
   await existingPersonRecord.save();
   res.json({ person: existingPersonRecord.toObject() });
 })
Reached this far without error
Error: Invalid query
    at model.Query.preFindHook
    at callMiddlewareFunction

To confirm this, we turned on debug mode in mongoose

mongoose.set('debug', true)
and removed the
pre-find
hook.

Mongoose: people.findOne({ _id: ObjectId("6031d9f91973241ccb4848e9"), age: { '$gte': 18 } }, { projection: {} })
Reached this far without error
Mongoose: people.findOne({ _id: ObjectId("6031d9f91973241ccb4848e9") }, { session: undefined, projection: { _id:

In debug mode, mongoose logs every query that it runs. It is clear that

findOne()
was executed here when
save()
was called. This
findOne()
call did not specify an age property in the filter. This made the
pre-find
hook throw an error.

The very idea of mongoose executing a different operation than the one it was asked to was baffling to me. I spent the next hour scouring the web for hints or explanations, finding none.

The mystery unraveled when I read the source code of

Model.save
as a last resort. The important parts are shown below. Complete source code can be found here.


Model.prototype.$__handleSave = function (options, callback) {
  if (this.isNew) {
    this[modelCollectionSymbol].insertOne(
      obj,
      saveOptions,
      function (err, ret) {
        if (err) {
          _setIsNew(_this, true);

          callback(err, null);
          return;
        }

        callback(null, ret);
      }
    );
  } else {
    const delta = this.$__delta();

    if (delta) {
      this[modelCollectionSymbol].updateOne(
        where,
        delta[1],
        saveOptions,
        (err, ret) => {
          if (err) {
            this.$__undoReset();

            callback(err);
            return;
          }
          ret.$where = where;
          callback(null, ret);
        }
      );
    } else {
      const optionsWithCustomValues = Object.assign({}, options, saveOptions);
      this.constructor
        .exists(this.$__where(), optionsWithCustomValues)
        .then((documentExists) => {
          if (!documentExists) {
            throw new DocumentNotFoundError(
              this.$__where(),
              this.constructor.modelName
            );
          }

          callback();
        })
        .catch(callback);
      return;
    }
  }
};

Mongoose internally tracks whether a document is ‘new’ or not. It also tracks the fields (if any) that have been updated on the document. This information is used to decide what operation is performed when the

.save()
method is called on the document.

  • When the document is ‘new’, mongoose performs an insert.
  • When the document is not new, mongoose calculates the ‘delta’.
  • If any fields have been modified, mongoose performs an update.
  • If no fields have been modified, mongoose checks that the document exists in the database using findOne.

What We Could Have Done Better

The reason why only the PATCH calls failed was that in a PATCH operation, which is a partial update, there is a possibility that none of the fields are updated. In this case, we should not have called

.save()
on the document in the first place.

The hook we wrote was flawed as well. The

_conditions
is a private property of 3rd party code. We should not be messing with it even if JavaScript allows us to.

In hindsight, it seems like mongoose was doing the sensible thing by simply ensuring that the document existed in the database.

Mongoose is a (heavy) abstraction layer over Mongo. It is incorrect to think of its operations as simple wrappers to those of Mongo’s native driver.

Tags

Join Hacker Noon

Create your free account to unlock your custom reading experience.