20,560 reads

Managing Complex Data Structures in NodeJS.

by Ethan JarrellSeptember 17th, 2017

Too Long; Didn't Read

When starting a new project, the question of how to properly organize your data is usually one of the first problems to solve. Let’s say that need to store data of users. Using Mongo, and storing the data as collections, we have a couple of options. As an example, let’s assume we need to store the following data about each user:

Company Mentioned

featured image - Managing Complex Data Structures in NodeJS.

User Information:

UserId
First Name
Last Name
Current Address
Email Address
Home Phone
Work Phone
Cell Phone
If the phone numbers have been verified

One way of handling this data would be to create a separate schema for each item on the list. Then, we could do a schema reference to reference data in the other schemas. That would look something like this:

const mongoose = require('mongoose');

let Schema = mongoose.Schema;

const userIDSchema = new Schema({

UserID: {type: mongoose.Schema.Types.Mixed,},})

const UserID = mongoose.model('UserID', userIDSchema);

module.exports = UserID;

Then, in our User First Name Schema, we would do this:

const mongoose = require('mongoose');

let Schema = mongoose.Schema;

const FirstNameSchema = new Schema({

UserID: {type: mongoose.Schema.Types.Mixed,

ref: 'UserId',

First_Name: {

type: String,

},})

const FirstName = mongoose.model('Firstname', FirstNameSchema);

module.exports = FirstName;

Then for each subsequent item on the list, we would do something similar. That way, each schema is separate, but would have access to the other data. However, this would get cumbersome when creating so many different models and routes. This is usually one of the main considerations when deciding how to organize data on the back end. How do you expect the data to grow, and does the current organization of data allow for easy expansion and manipulation of the data in the future? In our case of User Data, our current model might not be the best. Alternatively, instead of referencing the data between schemas, we could instead nest the data we needed inside objects and arrays within the model. Let’s reorganize our data like that:

const User = mongoose.model('Story', userSchema);

module.exports = User;

const mongoose = require('mongoose');

let Schema = mongoose.Schema;

const userSchema = new Schema({

UserID: {type: mongoose.Schema.Types.Mixed,},User_Info: {First_Name: {type: String,},Last_Name: {type: String,},Current_Address: {type: String,},Email_Address: {type: String,},},Phone_Numbers: [{Home_Phone: {type: Number,},Work_Phone: {type: Number,},Cell_Phone: {type: Number,},Phone_verified: [{Home: Boolean,

          Work: Boolean,

          Cell: Boolean,  
        }\],  
  }\],

})

const User = mongoose.model('User', userSchema);

module.exports = User;

Here, we have all of our user information stored in an object. Then, our three phone numbers are stored as objects in an array called “Phone_Numbers”. Inside the “Phone_Numbers” array, we have a sub-document called “Phone-Verified” which is an array of objects, indicating whether those phone numbers have been verified. As a model in mongoose, it’s easy to see how this data is organized and nested. However, correctly posting the data can be more difficult. If we are posting this data to an API, our response will be in JSON, and our API call might look similar to the following:

app.post('/api/user', function(req, res) {User.create({UserID: req.body.userid,User_Info: req.body.userinfo,First_Name: req.body.firstname,Last_Name: req.body.lastname,Current_Address: req.body.currentaddress,Email_Address: req.body.emailaddress,Phone_Numbers: req.body.phonenumbers,Home_Phone: req.body.homephone,Work_Phone: req.body.workphone,Cell_Phone: req.body.cellphone,Phone_Verified:req.body.phoneverified,Home: req.body.home,Work: req.body.work,Cell: req.body.cell,}).then(user => {res.json(user)});});

You may have noticed in our app.post, we have fields for “Phone_Numbers:” and “Phone_Verified” and “User_Info”, even though those fields in our model are just the name of arrays that contain no actual data. However, it’s necessary to include that in our app.post. Otherwise, the post won’t work.

Now, if we are creating a front end to our application, we’ll want to include this dependency in our app.js file:

const bodyParser = require('body-parser');

app.use(bodyParser.urlencoded({extended: true}));

When we call app.use, make sure we set it to “extended: true” This will allow us to properly post to nested data in our database from our front end form. That brings us to the last part of our puzzle, the front end form. Let’s tackle our UserID field first. That’s the easiest, since it’s not nested. Whether you’re using react, or mustache, or something else, the naming of our input would be the same. It would likely look something like this:

All we need to do is make sure the “name” matches the “req.body.userid” we used in our app.js file. Now, how do we post to the nested object of User Info? We would do something like the following:

Because we are using the body parser, and we have it set to urlencoded extended: true, the body parser will interpret this name field as a nested object. We would do the same thing for “last name” and the other fields within our nested object, like so:

Now the more complicated matter is our nested array. How would we post to that? Well, we would do something pretty similar, which would again be interpreted by our body parser. Let’s say there’s only one user. And that user only has one house phone, one cell phone and one work phone. In that case, our form input would look like this:

However, what if, in our data structure, you had multiple sub-users in each document, or multiple households, or family members who you needed to add phones for, but have them still be nested in the User. In that case, you could easily add an array of phone numbers. The input would change slightly on our front end.

Then our second input would look like this:

The body parser interprets this as the indexes of an array, and adds them as phone numbers in the phone numbers array inside our user collection. Now, as you can see, our verified phone number field is nested inside phone numbers. That would look pretty similar to what we’ve done here. It might look something like this:

As you can see, we are posting here, to deep nested data, and could continue this format with even deeper nested data. Although there are other ways to do this, this solution is elegant, easy, and might hopefully save some time. As a side note, this format also works in Postman, if you are posting your data that way.

Hope this helps, and feel free to reach out, if you have any feedback or comments. Thanks!