Site Color

Text Color

Ad Color

Text Color





Sign Up to Save Your Colors


Introduction to DevOps Automation Tools [ Hands-on Tutorial ] by@redalegzali

Introduction to DevOps Automation Tools [ Hands-on Tutorial ]

Reda Legzali Hacker Noon profile picture

Reda Legzali

IT Enthusiast

In this lab we are going to automate the process of creating the Infrastructure. We will also create a CI/CD Pipeline which will deploy a web application in a container-based environment. We're going to accomplish this using some interesting automation tools. Let's get started !


This lab is a step-by-step guide, so you should be able to follow along and copy paste the snippets and commands without any problem. However, having some knowledge of any of the following technologies will help you understand what is happening behind the scenes.

Linux CLI / SSH
Docker / Docker Compose


Before starting you have to first set up the following:

These tools are available for all platforms. For Windows users, just follow the links and download the exe file. Same for Mac OS users, follow the links and download the dmg file, although it is recommended to use Homebrew. Finally, for Linux users, just use your default package manager.


Alright! Now that your setup is ready, let's start by giving you some insights on what we are building.


This is the architecture of the project. We are going to use Vagrant to create these two Virtual Machines. The Manager Node will contain Ansible and will configure the 2 Virtual Machines using a Playbook. It will also contain Jenkins to setup the pipeline and Docker to perform the necessary builds and tests. The Dev Node will contain Docker to run the app using port 80.

You can find the demo application we are trying to deploy in this repository. As illustrated via the following architecture, it’s a web application with a Flask backend using a MariaDB database and communicating with a React frontend through a REST API.


Infra as Code

Enough "blah blah", show me some code!

That covers the basic theory behind the project. We can now start setting up the Infrastructure of the project. I will be using Linux commands so for Windows users please use "Git Bash" to follow along, it's installed with Git.

First, we need to clone the Infrastructure repository.

git clone

Now that the project is cloned you are gonna need to make some small changes. You should probably open this folder using your favorite text editor like Atom or Visual Studio Code.

Let's start by investing the Vagrantfile. You will find it here :


# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|

  image  = "bento/ubuntu-20.04"
  config.vm.define "manager" do |manager| = image
    manager.vm.hostname = "manager" "public_network" "private_network", ip: ""
    manager.vm.synced_folder "../Ansible/", "/home/vagrant/Ansible/",
    create: true, owner: "vagrant", group: "vagrant", mount_options: ["dmode=755","fmode=755"]
    manager.vm.synced_folder "../Jenkins/", "/home/vagrant/Jenkins/", 
    create: true, owner: "vagrant", group: "vagrant", mount_options: ["dmode=755","fmode=755"]
    manager.vm.provision "shell" do |s|
      s.path = "provisions/"
    manager.vm.provider "virtualbox" do |vb|
      vb.gui = false
      vb.memory = 2048
      vb.cpus = 2

  config.vm.define "dev" do |dev| = image
    dev.vm.hostname = "dev" "private_network", ip: ""
    dev.vm.provision "shell" do |s|
      s.path = "provisions/"
    dev.vm.provider "virtualbox" do |vb|
      vb.gui = false
      vb.memory = 1024
      vb.cpus = 1


The file is written in a programming language called Ruby. Basically, it will create two Virtual Machines running Ubuntu 20.04 inside VirtualBox. It will also automatically install Ansible inside the Manager server and mount the Jenkins and Ansible directories so that you can find them inside the server without doing anything. The only thing you can change in this file is :

depending on your system resources. Don't change anything else as you might break things :)

Next, let's check the Jenkins configuration file :


  systemMessage: Jenkins as Code 

    shell: /bin/bash

    - credentials:
      - usernamePassword:
          id: DockerCreds
          username: dockerhub_username
          password: dockerhub_password
          description: Credentials for Docker Hub
          scope: GLOBAL
      - usernamePassword:
          id: GitCreds
          username: github_username
          password: github_password
          description: Credentials for Git Hub
          scope: GLOBAL
      - basicSSHUserPrivateKey:
          scope: GLOBAL
          id: DevServer
          username: vagrant
          description: Private Key for SSH Agent
              privateKey: "readFile:${/var/lib/jenkins/.ssh/private_key}"

      - name: Default
        home: /usr/bin/git
      - name: Default
        home: /usr/bin/

This YAML file describes the required configuration for Jenkins. You must put your GitHub and DockerHub credentials in here. Make sure your credentials are correct and save the file.

And finally, there are a number of files inside the

directory. We will only have a look inside the
to have an idea of what's happening

- name: Setup System
  hosts: all
  become: yes
    - include: tasks/system.yaml

- name: Setup Docker
  hosts: all
  become: yes
    - include: tasks/docker.yaml
    - include: handlers/docker.yaml

- name: Setup Docker-Compose
  hosts: all
  become: yes
    - include: tasks/dockercompose.yaml

- name: Register Dev Server Private Key
  hosts: dev
    - name: Share Public Key
      shell: cat ~/.ssh/ >> ~/.ssh/authorized_keys
    - name: Register Private Key
      shell: cat ~/.ssh/id_rsa
      register: private_key

- name: Setup Jenkins
  hosts: manager
  become: yes
    private_key: "{{ hostvars[dev].private_key.stdout }}"
    - include: tasks/jenkins.yaml
    - include: handlers/jenkins.yaml

The file is pretty self-explanatory. In short, it does the following:

  1. Upgrade the system and install some useful packages like NGROK which we will discuss later.
  2. Install Docker and Docker Compose
  3. Fetch the SSH private key from the Dev server.
  4. Install Jenkins and place the Dev server's private key in the Jenkins home directory so that we can use it later.


Okay now that we have an overview of what this repository contains, we can resume our work. Get back to your terminal and navigate to the


cd MovieAppInfra/Vagrant

Now that you are on the right directory you can run the

. This will create our two Virtual Machines. It's an operation that will take a few moments the first time you run it, since it has to download the Ubuntu image.

vagrant up

If you have multiple network interfaces in your system you will receive this message. You have to choose your wifi or ethernet interface depending on what you use to connect to Internet. It's usually the first one. Press Enter after you have entered the number of your interface and let Vagrant do its magic.


If you open VirtualBox you should see your two Virtual Machines. You don't have to use VirtualBox at all. So you can close it. Your entire work will be done through Vagrant and SSH


You can always run one of these two commands if you wish to remove or shutdown the Virtual Machines. Just make sure you are in the right directory.

vagrant destroy // Shutdown and Erase the VMs
vagrant halt // Shutdown the VMs


Now that our servers are up and running, let's connect to the Manager server. Give it a second and you should be connected to the server. Your prompt will become :

[email protected]

Now let's navigate to the

directory. Remember that Vagrant mounted them automatically.

cd Ansible/

Now we need to share the Manager's public key to be able to run the Ansible Playbook. We will do so by running an Ansible Ad-Hoc command. This is the only time we are going to use a password. The password used in our demo is :


ansible -k -m authorized_key -a "user='vagrant' key='{{ lookup('file','~/.ssh/') }}'" all

To test that Ansible is working correctly, let's Ping Pong !

ansible -m ping all

We're all set.

Let's run the Playbook. Give it some time ( just like Vagrant ), it will take some time in the first time.

ansible-playbook playbook.yaml


First, let's retrieve the Jenkins initial password. To do so, execute this command.

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Store the key that this command returns somewhere or simply copy it as we will need it later in order to setup Jenkins.

Open your web browser and visit this repository, make sure you are connected to GitHub, then fork this project. If you are unfamiliar with the term forking in GitHub, it simply means that you will take someone's project and make it your own. Gotta love Open Source :)


This will redirect you to your repository. Now you should be able to modify the project.

  1. Click on the
  2. Click on the small pencil on the right to edit the file.
  3. Replace
    with your DockerHub username.
  4. Scroll Down and press Commit changes.

Now open a new tab in your browser and visit the following URL :

Paste the Jenkins password we copied earlier.


Then click on Install suggested plugins and give it some time. Don't worry if some plugins don't get installed, just continue we will go over the important plugins again later. For the next steps just enter your information and leave the default Jenkins URL. Jenkins will restart and ask you to login. Once you login, you will be redirected to the Jenkins dashboard.

Go to Manage Jenkins


Then to Manage Plugins


Make sure you are in the Available tab and install this list of plugins. This time if it fails you must try again.

  • Credentials Plugin
  • Git Plugin
  • GitHub Plugin
  • Pipeline
  • Docker
  • Docker Pipeline
  • Configuration as Code
  • SSH Agent

When all the installations are done, you will be redirected to the Dashboard. Click on Manage Jenkins again.

This time you should be able to see the Configuration as Code plugin. Instead of doing our entire Jenkins configuration manually, we will simply give this plugin our Jenkins.yaml file and let it handle the configuration for us.


Once you click on it, you should see this page. Just paste the path to the configuration file :


And then click on : Apply new configuration


You should get this message as a confirmation that the configuration has been set up successfully. You can go back to the Dashboard


We are now ready to create our pipeline. Go back to the Dashboard and Click on New Item.


Then select Pipeline and give your Job a name then press OK.


Start by checking the GitHub Project and enter your repository URL.


Next, enter the following parameters : registry = Your Docker Hub username remoteUser = vagrant remoteHost =


Then check this option so that the build can be triggered via GitHub Webhook.


And finally, tell the Pipeline to get the Jenkinsfile from the repository. Again, enter your Git repository URL. Select your Git Credentials which have been created with the configuration as code plugin. And finally change the branch to main. Now you can press Save and start the build.


Click on Build with Parameters and leave the default values. The build will take some time the first time you run it.


While it is running, let's talk a bit about what this pipeline does. First, here are the different stages



Creates a Python virtual environment and installs the dependencies.


Creates a temporary MariaDB container and run some Unit Tests in Python.


Builds the Docker images and pushes them in your Docker registry. If you go check your Docker registry after the Pipeline is completed, you will find these images.


Executes the Docker Compose in the Dev server which pulls the Docker images from the registry and starts the containers.


Now that the Pipeline is finished, give it a few moments the time for Docker Compose to settle and visit this URL :


And there you go! Your web application is deployed successfully.


Alright, we have come a long way. The last thing we are going to do is to create a GitHub Webhook. This will basically trigger our Pipeline every time we make a Push. Go to your repository in GitHub -> Settings -> Webhooks -> Add Webhook


Now go back to your Manager server. If you closed your SSH session or closed your terminal, just follow the previous steps to connect to the Manager server again. Once you are connected, execute the following command.

ngrok http 8080

You should get an output that looks like this. Copy the HTTPS link in my case it's : 
This is a temporary link, the purpose of this is to simulate a public ip address. You wouldn't do this in a production environment.

Now go back to GitHub. Paste your NGROK URL followed by

For my case
Choose application/json and leave the rest as default and Click on Add Webhook.


You will be redirected. Refresh the page and if you get this green checkbox you are good to go.


Now let's make a Push to our repository. Just modify the README file inside GitHub and Commit changes.


Boom ! Our pipeline is triggered automatically.


This wraps up our lab. From here you can extend this pipeline and add one or two more Dev servers and use some container orchestrator like Kubernetes. You can also monitor your infrastructure and application with some tools like Prometheus or Elastic Search. And don't forget about security, for the sake of simplicity we used plain text passwords in this lab but you should probably use some secrets management tool like git-secret, git-crypt or Vault.


Samy Simon

Allaki Driss