A new Continuous Integration Coordinator written in Elixir & Phoenix I have been wanting to learn Elixir and for a while now, and a couple of months ago I finally took the plunge. I started reading “Programming 1.3” by Dave Thomas, and “Programming Phoenix” by Chris McCord, et. al. (both from “The Pragmatic Bookshelf” and highly recommended if you want to get started with Elixir and Phoenix). Phoenix Elixir I followed most of the code guides in the books, but it still felt kind of foreign sometimes. I wanted to build something a bit more complex, and challenging. I also wanted to release it as open source software, so it had to be useful to a wider audience than just me. I’ve always liked CI systems and the value they provide, but I have yet to find one that offers everything I need. I want it to be highly configurable, easily scalable, easy to install, with a robust executor that can run on multiple operating systems, and to be . The CI service that comes closest to this is GitLab CI, but it is tightly coupled with GitLab itself, so I cannot use it with GitHub. I also don’t want to install a whole Source Code Management system for it’s CI service. open source just That is where the idea to create a CI system came from. I learned a lot about how CI systems should work from my time at GitLab, so it felt like the right step forward. In fact, I decided to use the GitLab CI Runner as the executor for this project. The GitLab CI Runner is written in and is the piece of code that actually takes care of running the builds, and whatever other action you need in order to execute your pipeline. The greatest thing about it, is that it is basically a , meaning that all you need to do is tell it what to do in the way it expects, and it will do it! It is also incredibly powerful and configurable, it can use Docker to execute your builds, and takes advantage of Docker Machine to let you autoscale the machines it uses for your builds, it can use Kubernetes, run on top of Digital Ocean’s VMs, run Xcode and Windows builds, and a lot more. Go “dumb box” Using an already existing project as the executor for my CI system allowed me to concentrate on the coordinator, which is the piece of software that reads your project’s configuration, creates the pipelines, and builds, and sends all this information to the executor in the way it expects it to be. It also collects all the information relayed by the executor, like build trace and other artifacts, and presents the to the user in an easily digestible manner. Since this is an introduction post, I will not go into details of the implementation of the different parts of AlloyCI. For that I will create and will go into as much detail as possible. For now, let me tell you about its features. several separate posts (Part 1 of the ) AlloyCI Dev Diary is live! Features Please keep in mind that this is the first public release of AlloyCI, and it is only at version 0.1.0. A lot of the features will improve over time and a lot more will come as we move towards 1.0. You can check the roadmap to 1.0 here: and if you’d like to contribute anything, please go ahead. https://github.com/AlloyCI/alloy_ci/projects/1 Basic CI functionality For version 0.1 the goal was to have at least the very basics of CI functionality. These include the following: Parse a configuration file to create and prepare the builds and pipelines Support different build stages Allow builds and runners to be tagged, thus allowing for fine grained control of which runner picks up which build Process and send the build information to the GitLab Runner in the expected format Accept information received from the GitLab Runner about the specific build Show the output trace of each build and follow it when looking at an active build Properly manage the status of each build, and send to the Runner only builds that should be processed Report to GitHub the status of each push via the reference of its last commit SHA Send notifications to the user about the status of a pipeline via email and/or Slack Perform auto-deploys via the stage deploy As commercially available CI systems go, AlloyCI is dwarfed by the plethora of features other CI services offer, but that’s OK. I wanted to start simple and distill the CI experience to only its most necessary features. Bells and whistles will come later 😉 Extras Something I did add just for the “nice to have” value are build statistics per project and runner. You can see some nice charts portraying the amount of build in the past week, how many have succeeded or failed, and the total of builds processed. Project’s Settings The runners chart will also brake down the amount of builds processed by project, so you can see which runners are picking up which builds. Runner’s Admin Settings How can you try it out? You can install AlloyCI on your own servers using Docker, and Docker Compose. Please follow our to get started. documentation The easiest way to try AlloyCI out is by going to , logging in with your GitHub account, and adding a project. https://alloy-ci.com New Project’s View Before being able to add a project, you will be prompted to install the AlloyCI integration on your account. After doing this, the project you’ll want to add needs to have an configuration file before it can be added. You can read more about how to create this file . Here is a quick example for Elixir: .alloy-ci.json here {"image": "elixir:latest","services": ["postgres"],"before_script": ["mix deps.get"],"after_script": ["rm secrets"],"stages": ["build","test","deploy"],"mix credo": {"stage": "build","script": ["mix credo"],"tags": ["elixir"]},"mix test": {"stage": "test","script": ["mix test"],"tags": ["postgres", "elixir"]}} Once the file has been committed to your repository, you are ready to add it to AlloyCI. After the project has been added, and new code has been pushed, you will see the newly created pipeline in the project’s dashboard. Project’s Dashboard If you click on the pipeline, it will take you to the its builds, where you can follow their output and overall status. Build Job’s Output The test server has limited resources and only has 2 registered runners to process builds, so it will take a while to process your pipeline. Luckily you can register your own runner as a “Project Specific Runner”, which means it will only pick up builds from your own, particular project. Go to your project’s settings for instructions on how to add a specific runner. Why did I build AlloyCI? As I mentioned before, I built AlloyCI to learn Elixir and Phoenix. I chose a CI coordinator, because I’m familiar with them, and wanted to add my own take on them. I’m also releasing it as licensed open source code, because I think other people might find it useful. Having a non-trivial codebase to look at is a really great resource to learn a new language. I know my code is far from perfect, but it is a start, and I would love for people to look at it, criticize it, and provide useful feedback, so that I, , can learn and benefit from this project. GPLv3 and others I will continue to develop AlloyCI for quite some time. I plan to maintain it for the foreseeable future, and I might even consider writing a new runner/executor in Elixir itself, so that the project does not rely on GitLab’s good intentions of keeping the GitLab Runner open source and maintained. — If you like the project, please star it on GitHub, and if you think of something the project could use, or have some suggestion or feedback, by all means, please open an issue. I would ❤️ to hear what you think.
