Terraform module for automated MongoDB backup

MongoDB is one of the most widely used databases out there, and creating backups for it is a crucial, yet routine task not to be taken lightly. This is why we decided to automate the process.

Manual backups are utterly outdated, not to mention this means keeping in mind all the peculiarities and tagging the copies by hand. Automated backup requires using certain libraries that are not present by default, and the DevOps team will most likely learn of this the hard way.

Thus said, we wanted to reach the following results while creating a Terraform module for automated MongoDB backup:

• Automated management of the backup process (you only need access to your AWS account, installed Terraform and our solution)
• Creating a structured and easily-accessible registry of backups (storing several latest versions of backups, ensuring they are available from multiple access points)

Screenshot 1: AWS CloudFormation interface with a list of periodic tasks

Working with the automated MongoDB backup tool from IT Svit

We currently have a Terraform manifest in place, that installs all the needed libraries and dependencies, enabling the automated backup to AWS S3 cloud storage, using the AWS CloudFormation and Data Pipeline tools. CloudFormation works as a scheduler, starting the Data Pipeline task that creates a MongoDB dump as a backup and stores it in the S3 bucket. The process logs can be accessed through Terraform or through the Data Pipeline web interface.

Screenshot 2: The list of backuping tasks in AWS Data Pipeline

Screenshot 3: Database backups stored within an AWS S3 bucket

Screenshot 4: Each backup is versioned, meaning a specific version of any file can be restored if need be.

All in all this is a neat little tool solving one of the major headaches of any DevOps team: automated database backuping, backup version monitoring and simple recovery upon request.

So, the only things needed to use the solution is access to AWS services and a Terraform installed on your machine. Terraform runs the manifest that installs the needed libraries and configures the backup workflow. Check it out on Github!

This story was originally published on my company’s blog — https://itsvit.com/blog/automate-mongodb-backups-using-terraform/