with Ansible, Vagrant and Continuous Integration Problem Developers need a specific set of software tools to get the job done. You might have in your team you need to fundamentally change the software environment that your current team uses in some way — new starters OR perhaps migrating to a different OS? The list of software tools we all need can be quite . While some of these software tools you can obtain from package managers (e.g. Aptitude, Snap, Homebrew, etc…), they can still have quite complex installation steps extensive (which could randomly change and break!). These steps might be , they might not. There’s also the problem that having a developer maintain their own base software environment can mean huge inconsistencies between different dev machines that get worse over time. documented Addressing the problem can create a huge hit to productivity. For on-boarding new starters it can mean a whole lot of effort lining up knowledgeable people in your team supporting your new starter for the next X days, or a long and drawn out migration plan if your problem is needing to change a dev team’s existing software environment. A Solution! Leverage the power of and script up your development environment (with all of its tools and dependencies) in a playbook. Ansible Use to spin up a of your base machine image Vagrant VirtualBox (of the OS / Distro you’re using). Run Ansible as a provisioner in the build of the virtual machines and use the playbook you’ve wrote. Output success or failure of provisioning steps and run in a Continuous Integration tool. Communicate updates and releases of this playbook to your team and ask them to pull and run the updates. What do you get? A base development environment that will be consistent across your team, that you know will have everything a developer needs to get started and you can rely on every time it’s installed . (providing it’s actively maintained) What the heck is Ansible? is an IT automation tool which can be used for a number of different tasks and IT needs. It uses a declarative DSL written in YAML and can be run locally, within the provisioning of a virtual machine, cloud environment or via ssh against a list of known hosts. Ansible Writing a Playbook For the sake of brevity for this article, I’m going to assume some Ansible knowledge — there’s a fantastic zero-to-hero tutorial below which will take you through the Ansible syntax and some of the built-in plugins it uses: _Updated for Ansible2! Ansible is one of the simplest server provisioning and configuration management tools. This is a…_serversforhackers.com An Ansible2 Tutorial We’re going to start with a basic playbook file which will reference the roles and tasks we need to create a development environment: We’ll then build out the roles / tasks and vars we need. I’ll only really describe the first of these — you can if you would like to see how the rest of the steps are implemented and the project should be structured. checkout the playbook Packages Task Vars file for Packages Task Running the Playbook ansible-playbook -i "localhost," -c local configure.yml [00:00:30][00:00:30] PLAY [all] *********************************************************************[00:00:30][00:00:30] TASK [Gathering Facts] *********************************************************[00:00:31] ok: [localhost][00:00:31][00:00:54] TASK [packages : Install required apt packages] ********************************[00:02:59] changed: [localhost][00:02:59][00:02:59] TASK [packages : Remove unrequired apt packages] *******************************[00:02:59] ok: [localhost] If any of the steps fail, the playbook fails and returns an exit code of 0 (failure). Stitching it Together (and Testing) Great! We now have a playbook we can use to install and remove various aptitude packages — but as we extend it, how should we run it and test it to make sure it ? doesn’t brick our machine We don’t really want to risk breaking our current environment if we can help it. This is where Vagrant comes in handy. What the heck is Vagrant? is an open source tool for initialising and configuring virtual environments. Vagrant provides a command-line interface for managing virtual environments, and uses a file called a to provide the definition of the environment that needs to be built. Vagrant Vagrantfile Here’s a very simple which creates a VM containing an Ubuntu OS: Vagrantfile We can configure a to run our Ansible playbook once a virtual machine of our choosing has booted up: Vagrantfile Bringing our Development Machine Up take great pride in how simple it is to bring up a Vagrant machine — simply: Hashicorp vagrant up Bringing machine 'dev' up with 'virtualbox' provider...==> dev: Checking if box 'bento/ubuntu-18.04' is up to date...==> dev: Running provisioner: ansible... dev: Running ansible-playbook...PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit="dev" --inventory-file=/home/andy/Projects/development-environment-playbook/.vagrant/provisioners/ansible/inventory -v configure.ymlUsing /home/andy/Projects/development-environment-playbook/ansible.cfg as config file PLAY [all] ********************************************************************* TASK [Gathering Facts] *********************************************************ok: [dev] (If it’s the first time you’ve done this, you’ll see a lot of output from the downloading of the appropriate VM image if it’s not available on your machine already). Integrating Playbook Build into a CI Pipeline You can probably guess what would come here if you’re familiar with the concept of and the tools available. There will be many options and differences here depending on what CI solution you use. In this example I’ll briefly cover over how this could work in one of the more popular solutions: — Continuous Integration Jenkins which I’ll also assume a working knowledge of — here’s a good knowledge-base around Jenkins pipe-lining . Installing / Configuring Jenkins I’ll assume a pre-existing installation of Jenkins — but if not, here’s the official docker image so you can try it out: _Docker official jenkins repo. Contribute to jenkinsci/docker development by creating an account on GitHub._github.com jenkinsci/docker The only special considerations here are to ensure that the Jenkins CI server, container or VM has all the dependencies it needs to run VMs via Vagrant. This will obviously differ dependent on the base OS and version. Here’s an example for installing Vagrant on Centos 7. A Simple Pipeline Assuming you actually want to know when your build fails, with the help of the — your could be as simple as: Slack plugin for Jenkins Jenkinsfile pipeline {stages {stage ('Start') {steps {sh 'vagrant up'}}}post {success {slackSend (color: '#00FF00', message: "A new version of the development environment is available!")} failure {slackSend (color: '#FF0000', message: "The latest build of the development environment FAILED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")}}} And if all goes well? Voila! Rolling it out If you commit to this approach and embed this in your team — here’s what benefits you could see: You now have a comprehensive list of most of the software the team is using. You can on-board new starters much more quickly because all they need to do is run an automated playbook to have a working environment. When “developer accidents” happen and machines are irrevocably broken — a new and clean state isn’t far away and can be achieved in a short space of time. You can roll out updates of a standard development environment by creating a new release version of the playbook. If install steps change or become outdated, the build will fail and you will get early visibility that you need to update your steps before it becomes a problem to solve at an inconvenient time. Other OS’s? I’ve focused quite a lot on the specifics of implementing this strategy for a Linux based environment , but this strategy can really apply to any OS . Providing you have good scripting skills in your preferred environment — you could easily tailor this strategy to suit your needs. (specifically Ubuntu) (in principle any that you could obtain a VM image for) Final Thoughts (and an example) There are obviously many ways to achieve the same result as this, but hopefully this article will help those in need of a solution to this problem and who perhaps have constraints preventing them from adopting other approaches. Here’s a full example I’ve whipped up for a development setup I’ve used in the past. It’s by no means perfect. _Ansible playbook for an ubuntu development environment. - AndyMacDroo/ubuntu-development-environment-playbook_github.com AndyMacDroo/ubuntu-development-environment-playbook Note: The cloud based CI integration I’ve used here to build the project, unfortunately doesn’t use VirtualBox / Vagrant due to some known limitations in this area (in a nutshell it amounts to issues running and deploying VMs inside the virtualised workers initialised in cloud CI pipelines)— the configuration I’ve applied here instead just runs the playbook against the Ubuntu Bionic VM provided by AppVeyor. Ideas and Further Suggestions: Add replacements to to include any custom scripts or utilities that developers might need or would save them time into your playbook. .bashrc Add additional tests / health checks to confirm vital software and tools have been installed correctly and are functional to improve reliability of the CI build. Create CI build steps to run a build where each successive release of your new and shiny development environment playbook is applied cumulatively over your virtual machines in test (dev machines are rarely clean for very long — ensure your playbook can be used for ‘updates’ of an already existing environment as well as from clean builds). Split your playbook into a base playbook and a development environment specific one — if you’re tempted to use Ansible to provision your production environment, why not make sure that your team’s development machines also shares things in common with your production environment? Use a technology like to create an image of the development environment and build a CI process around running the image rather than spawning virtual machines. Packer Thanks for reading! Hope you enjoyed it☺️
