Combining Technologies: MongoDB & NodeJS

In this overdue blog post, I’m going to walk through the process of creating a basic stat-tracking app. In this post, I’ll cover the following technologies:

1. Node.js
2. Mustache.js
3. Mongoose
4. MongoDB

What we want to do, is use node to create api endpoints, which we can use to read and write to our Mongo Database. We will use Mustache to display information from the Database on the specific pages we need.

You might be asking yourself, “what would I use such an application for?” Well, as a completely hypothetical example, let’s say that you just graduated from an intense 12 week, programming bootcamp, and need an efficient way to track the resumes you send out and interviews you’ve had.

Before we actually begin any coding, it’s a good idea to whiteboard out what we want it to look like so we can visualize what input we’ll need, and what our database structure should look like. Where’s what I’m envisioning, and what I’ll try to create in this post:

I’m thinking basically three columns, with our category input on one side. When we input a new category, I want it to dynamically appear below the input. We’ll create an API endpoint for this, and then display it through Mustache. We also want an activity to be associated with a category. Since we’re using a document based database, this can be tricky. So, to make life simpler for us, we’ll force the user to either select a category, or input a new category, before creating an activity. That way, we can grab the category from the parameters once we reach that second endpoint. Finally, we’ll have activities listed in the final column. What I’d like here, is to have all the activities we’re tracking to be listed here. And then, when a user selects a category, it filters those results and only shows the activities listed for that category. We’ll need to create a find() query in that endpoint to filter our results, and then also change what mustache is displaying in our view.

The way I see this, is having all the content visible at the same time, but only allowing the user to interact with one column at a time. That way, the user never feels like they’re actually navigating to a new page. We’ll do this partially through the endpoints, and also mustache. On the third panel here, I’d like a text based approach to displaying the data for each individual activity. Something similar to the far right column.

And here’s what our simple database structure should look like. Eventually, we’ll want to associate new users with categories, and then associate categories with users. This is a pretty simple 1 to 1 relationship. Document based databases can be difficult to work with when creating relationships, so it’s a good idea to plan out your project like this before you begin. If it looks like you’re going to need a lot of relationships and associations, it may be a better idea to use a relational database instead.

I’ll also include a link to my github, and a video once I’m finished, so you can see what the finished project looked like for me.

Okay, so let’s get started. Generally, I like to start by creating my database first. If you haven’t done so yet, go ahead now and install MongoDB. You can find the link here: https://docs.mongodb.com/getting-started/shell/installation/

Install and Start Mongo

Once you have it installed and have started MongoDB, connect the mongo shell to your running MongoDB instance.

You can then run the Mongo Shell with the following command line prompt:

mongo

Now, if you have any databases already saved, you can view them by typing:

show dbs

Next, you can either create a new database, or use an existing one. The command is the same in either case. I want to create a database for my stats, so I’ll create a database called stats.

use stats

In mongo, if there is a database called stats, it will switch to that one. If there isn’t an existing database by that name, it will create one. It’s that simple. You should see something like this:

switched to db stats

Now, we can see any collections we have in this database by using the command:

show collections

We don’t have any collections yet for our stat tracker, so we’ll make a few new ones. Collections are like tables in relational databases, although they lack the ability to easily make associations between collections, like you may be used to, if you’re more familiar with SQL databases. For our stat-tracker, we know we’ll need at least 3 collections:

1. Users, so we can log in and out as different users.
2. Categories, because we may want to track different categories of stats.
3. Activities. Activities would be specific activities associated with each category. This may seem like an additional step, but I think it will help us keep things organized. Inside our Activities collection, we’ll keep track of data, such as activity name, quantity of activity completed, date of completed activity, etc.

Here’s how we’ll create these collections:

db.createCollection("users")db.createCollection("categories")db.createCollection("activities")

Now that we’ve created them, we can view our collections by using the command:

db.activities.find().pretty()

The .pretty() isn’t necessary, but it makes it easier to view the data in our collections. If you like working in terminal, then you can continue to go back and type that command into terminal, which is a good habit to get into, to make sure the data you want is being correctly added to the database. However, I’m a very visual person, and like to be able to see the data in a more visually appealing way. There are programs out there for Database visualization. Robo3T is an okay one, but I also just discovered a product that’s made by MongoDB called MongoDB Compass. It’s by far my favorite technology for database visualization, so if you’re into that sort of thing, I would recommend going over and downloading it. You may also find some other great apps they have like atlas, for DB deployment. Here’s a screenshot of what you can see with Compass:

Set up our App.Js Environment

Now that we’ve got that done, we can do the rest of the work in our text editor. As you’ve probably noticed, I like step by step lists. So let’s make a list of what we’ll need to do.

1. Create an app.js file where we’ll store all of our routes.
2. Make Models for each of our collections.
3. Create JS files for each of our Models, and import them into our app.js file
4. Create Mustache.js files where we’ll view all of our data and information.
5. Create a CSS file to style our views.

Lets start with our app.js file first. Here, we’ll list our dependencies, and then start on our routes. Here are the dependencies I think we’ll need:

const express = require('express');const parseurl = require('parseurl');const bodyParser = require('body-parser');const path = require('path');const expressValidator = require('express-validator');const mustacheExpress = require('mustache-express');const uniqueValidator = require('mongoose-unique-validator');const mongoose = require('mongoose');const session = require('express-session');const app = express();

Next, we’ll need to require our models as well. We haven’t created them yet, but we can go ahead and require them now, and just make sure we use the same names when we create and export them.

const Activity = require('./models/activity');const Category = require('./models/category');const User = require('./models/user.js');

Next, we’ll set up our views engine and the app.use for our Body Parser and public(CSS) files:

app.engine('mustache', mustacheExpress());app.set('view engine', 'mustache');app.set('views', './views');app.use(express.static('public'));app.use(bodyParser.json());app.use(bodyParser.urlencoded({extended: true}));app.use(expressValidator());

Connect to Mongo

Now we’ll need to connect our app.js file to our mongoDB.

mongoose.connect('mongodb://localhost:27017/stats')

We named our database ‘stats’, so make sure the name is the same here as well.

Next, we’ll create an app.get for our root directory ‘/’, and an app.use to test our connection. It’s good to write a console.log message here that let’s you know you’re connected.

app.get('/', function(req, res) {res.redirect('/api/splash');});

app.use(function(req, res, next) {console.log('Great Job! You connected, you charming, handsome fellow!');next();})

I usually like to include an encouraging message here as well. As you may find out soon, things often don’t work, so it’s nice to have a little encouragement here.

At the bottom of our file, lets go ahead and create an app.listen, so we can make sure everything is connected, before we proceed. This will go at the very bottom of our app.js, and we’ll include an export for our app at the very bottom. It should look like this:

app.listen(3000);console.log('starting applicaiton. Good job!');

module.exports = app;

Building our Mongoose Models

Now, let’s think about the pages and functionality we will need, so we can begin creating our endpoints.

1. A landing page. Here the user either login or signup. Once logged in, we’ll begin a session so that we can save the information.
2. A signup page. Here we will create a new user and add that user to the database
3. A Login page. Here we will start a session, and associate any created activities with that user.
4. Create a category page
5. Create an activity page
6. View statistics page. Here we’ll want to be able to view specific data about each activity.

We could add other features, or other pages if we wanted to, but this should get us started for now. Let’s start with our Category and Activity creation. In order to do that, we should probably build our models first. Let’s take a look at Activities first.

const mongoose = require('mongoose');const uniqueValidator = require('mongoose-unique-validator');

let Schema = mongoose.Schema;

const activitySchema = new mongoose.Schema({

category: {type: String,ref: 'Category',},activity_name: {type: String,},quantity: {type: Number,},metric: {type: String,},date: {type: Date,default: Date.now},});

const Activity = mongoose.model('Activity', activitySchema);activitySchema.plugin(uniqueValidator);module.exports = Activity

So there are a few things to point out here. First, let’s look at the top and bottom of our model. We start off by requiring mongoose. Then we call the variable Schema, and set it to mongoose.Schema. Then we create our activitySchema based on the mongoose.Schema.

At the end of the file, we basically let Activity be the const that we are exporting, which is all the data contained in our model. Remember, Activity, was one of the models that we required in our app.js file, so make sure that we’re using the same name here.

In between, is all of the data we want to include in our model. Our model is fairly simple, as each item we have really only has one piece of information, which is the data type. We could write this section more simply, like the following:

activity_name: type: String,

quantity: type: Number,

metric: type: String,

But creating an object from each item allows us to expand, and add data later to each object, if we need to. You may have also noticed that we have a category field here, even though we are going to make a Category Model.

category: {type: String,ref: 'Category',},

This includes a type, and a field called ‘ref:’. This references our Category model, and will actually bring data, specifically the category name, into our Activity model. You could embed this data in the activity model, but sometimes it’s handy to have them separate. Okay, let’s create our Category Model.

const mongoose = require('mongoose');const uniqueValidator = require('mongoose-unique-validator');

let Schema = mongoose.Schema;

const categorySchema = new mongoose.Schema({

activity_type: {type: String,},

stats: [{type: String,ref: 'Activity',}],});

categorySchema.plugin(uniqueValidator);const Category = mongoose.model('Category', categorySchema);

module.exports = Category

As you can see, our Category model is set up exactly like our Activity model. Here again, we want to make sure the names we’re using in app.js match the name we export here, ‘Category’. Also, I included a field here called stats, where I’m also referencing an array of data from our Activity Model. We may also want to edit our data type here. Instead of string, we might rather have mongoose.Schema.Types.ObjectId

And, just as fair warning, what we are trying to do here is make associations between collections in a document database. This is not an easy thing to do, and often will not work as you intend. Generally, you only want to do this when you are making simple one to one associations. Document based databases can become difficult to work with when trying to make multiple associations or many to many associations.

Since our association is simple, it’s probably okay. But if your project requires much more than that, it’s probably going to be much less of a headache to use a SQL database like Postgres.

Okay, now, lets go ahead and make our Users model. It should follow the same pattern.

const mongoose = require('mongoose');

let Schema = mongoose.Schema;

const userSchema = new Schema({username: {type: String,},password: {type: String,},category: [{type: mongoose.Schema.Types.ObjectId,ref: 'Category'}],})const User = mongoose.model('User', userSchema);

module.exports = User;

Here again, I’m referencing the category model, so that we can associate the category with the user collection. Now that we have our models, we can go back into our app.js file and being making our api endpoints.

Building our API Endpoints

1. Create a New Category

//==CREATE NEW CATEGORY=//

app.post('/api/home', function(req, res) {Category.create({activity_type: req.body.category,}).then(activity => {res.redirect('/api/home')});});

Here’s our first app.post, which is what we need to use to create something. When we create our forms on our mustache view, we’ll need to make sure the forms also have a method=”post”.

This endpoint will be /api/home, or basically our root page. If someone goes to ‘/’, we can redirect them to home. As you can see, after a Category is created, we are using res.redirect to send the user back to the home page, or /api/home.

Here, we have used the mongoose ‘create’ method, which builds a model, and saves it all in one. We are basically saying, “Hey Node, remember our Category model? Yeah, so, we have a field called activity type. And we want to populate that field with data. The data we’re using, you can find in the body of our document, in an input field with the name, ‘category.’”

Now, we’ll need to remember the name ‘category’. When we build our mustache view, we’ll need to use this name as the name for our input field, so that they can properly connect with each other.

2. Render the Category Home Page.

//==RENDER HOME PAGE=//

app.get('/api/home', function(req, res) {User.find({}).then(function(users) {Category.find({}).then(function(categories) {Activity.find({}).then(function(activities) {console.log(activities);res.render('home', {users: users,categories: categories,activities: activities,})});});});});

This is our app.get, which is what we need to use in order to render data instead of posting data. We also have a res.render at the end, and ‘home’. When we create our view in mustache, we’ll need to make sure we name it ‘home.mustache’, so that it corresponds to this res.render method, and can render the correct page.

Here, we are using a User.find, Category.find, and Activity.find. But on the home page, I want to display all the categories, and all the activities, so I haven’t put any queries between the ({}) in any of the finds, to make sure I get all the data and display all the data on this page.

3. Create an Activity

app.post('/api/:activity/:_id', function(req, res) {Activity.create({activity_name: req.body.activity,quantity: req.body.quantity,metric: req.body.metric,category: req.params.activity,}).then(activity => {console.log("about to log categories");res.redirect('/api/:activity/:_id')});});

Our second app.post is a little more complicated. our endpoint has more paramaters, for activity and id, because we’ll likely need to pull data from our paramters here so that we can populate some of these fields. We are again using the mongoose create method. We’re also telling mongoose basically the same thing we did in our create Category. We’re telling it, “Hey mongoose, remember the activity fields in our Activity model? They are labeled activity_name, quantity and metric. Anyway, you can find those in the body of the document with corresponding input field names of activity, quantity and metric. Category though, you’ll need to get that from params.”

Now, after creation, we’ll redirect the user back to the activity page with the endpoint /api/:activity/:_id.

One side note. You may have noticed that in our Activity model, we have a date field, but here, we are not asking for a date input. That’s because in the date object, we have included a field called ‘default’. Default basically says, if there is no date input, use todays date, or ‘date.now’. You could also write this using ‘timestamps.’ That would look more like this:

date: {timestamps: true,}

date.now also gives you a timestamp. However, the timestamps method in mongoose gives you an automatically created date and also an updated date. So depending on what you need, you could use either one.

4. Render the Activity Page.

//==RENDER ACTIVITY PAGE=//

app.get('/api/:activity/:_id', function(req, res) {User.find({}).then(function(users) {Category.findOne({activity_type: req.params.activity}).then(function(categories) { Activity.find({category: req.params.activity}).then(function(activities) {res.render('activity', {users: users,activities: activities,})});});});});

We’re using app.get here, and the same endpoint as our post, because we’ll also need to pull parameters here for our queries. In Category, we’ll use the findOne method, because we’ll only need on category at a time here. We’re inputting an activity to one category. Then we tell it where to find the activity we want, which is in req.params.activity. We then do an activity.find as well, where we’re finding all the categories, associated with the activities in the params. This may sound a little confusing. But think about it this way. Let’s say we have a category called, “Finding a Job” . and within that category, we want to display it’s activities, like “resumes sent”, and “interviews”. Instead of doing a query for the category model, we can just includ that here in the activities mode. Since we’ve include a reference to the categories in our activities model, we now have that information in activities as well.

Here, we’re rendering ‘activity’. So let’s make ourself a reminder that when we create our mustache view, we’ll want to make sure the name of our file matches the render location here, so we render the correct page.

5. Render a Specific Activity

//==RENDER SPECIFIC ACTIVITY=//

app.get('/api/:activity', function(req, res) {User.find({}).then(function(users) {Category.findOne({activity_type: req.params.activity}).then(function(categories) { Activity.find({ activity_name: req.params.activity}).then(function(activities) {res.render('date', {users: users,categories: categories,activities: activities})});});});});

Here is another app.get with a new endpoint for this page. Here, we want to display specific data about an activity. We are again pulling that data from our Activity model with Activity.find, and then querying that data based on the parameters we are passing it.

Creating the Views with Mustache.JS

Now, after all of this, we might like to do different things, like deleting or updating an activity. But given the length of this post, and the attention span of most readers (myself included), I won’t get into that here. The last thing I do want to show, is how these api calls line up with our forms. when you create your mustache files, you’ll again want to name them with the same names that we’ve used in our res.render calls.

Then in our forms, we’ll use the method=”post” and use the same names for the input fields there, as we’ve called for in our Activity.create and Category.create. Here’s a sample of the code in my mustache view for the Activity.create post:

<div class="activity_input"><form action="{{#activities}}{{category}}{{/activities}}" method="post"><div class="title1container"><div class="title1">Step 3: Use the Input Fields to Add an Activity to the Category You Selected</div></div><div class="formcontainer">

<div class="formInput">

<div class="labelAndCheckbox"><label class="title2">Category Name:</label>

<div class="act">{{#categories}}{{activity_type}}{{/categories}}</div>

</div>

<div class="title2">New Activity</div><input autocomplete="off" class="input" required type="text" name="activity">

<div class="title2">Quantity</div><input autocomplete="off" class="input" required type="text" name="quantity">

<div class="title2">Unit of Measurement</div><input autocomplete="off" class="input" required type="text" name="metric">

<input class="form__submit-button" type="submit">

</div></div></form>

Now, there is a lot going on here. But you can see in each input, we’re using the same names we use in our api call. “metric”, “quantity” and “activity”. We are also including this bit at the top of the form:

<form action="{{#activities}}{{category}}{{/activities}}" method="post">

which tells mustache that it’s a post, and where the action is “category”. All of our other forms will follow this same method. In our mustache view, one other tricky piece of the puzzle is figuring out how to address the links from one page to another. You’ll want to use mustache to call out the link location, so that those links are dynamic. Here’s an example from the home page to the activity input page:

<label class="title2">Category Name:</label>

{{#categories}}<div class="act"><a class="categoryList" href={{activity_type}}/{{_id}} value= {{_id}}{{#dates}}{{date}}{{/dates}}>{{activity_type}}</a>

</div>{{/categories}}

the href value is actually the {{_id}} of the {{#categories}} database. This can be a tricky thing to figure out, but once you go through it a few times, it’s not that bad.

On most our pages, we’re only displaying the activity or category name. But on our final page, we’ll want to actually display almost all of the information about the activity. This can also get a little confusing, since there is so much data, and displaying it in an organized way, with mustache, can be a little confusing at times. Here’s how mine looked in my mustache file:

{{#activities}}

<div class="categoryTitle">

  </div>

<div class="activityDateContainer"><span>While doing</span><span class="bold">{{category}}</span><span>stuff,</span><span class="bold">{{#users}}{{username}}{{/users}}</span><span>completed</span><span class="bold">{{quantity}}</span><span class="bold">{{metric}}</span><span>of</span><span class="bold">{{activity_name}}</span><span>on this date:</span><span class="bold">{{date}}</span></div>

{{/activities}}

Now, this may not make a lot of sense. If you’re a visual person like me, it helps to be able to see what it’s actually going to look like. To help with all of that, I’ve included a short video, where you can see how the site actually works. I hope this helps. And if you have any questions, or I’ve got something wrong, feel free to reach out. Thanks!

Ethan Jarrell | Professional Profile | LinkedIn_View Ethan Jarrell's professional profile on LinkedIn. LinkedIn is the world's largest business network, helping…_www.linkedin.com

ethanjarrell/stattracker_Contribute to stattracker development by creating an account on GitHub._github.com