“The advantage of feature branching is that each developer can work on their own feature and be isolated from changes going on elsewhere.” ( FeatureBranch ) Feature branches have been around for a long time and are a common practice among dev teams. I couldn’t imagine development in a team without them. It would seem ludicrous if someone asked me to who is working on a different feature. Just the thought makes me uncomfortable. It’s easy to see how the isolation that feature-branches provide, is also needed for testing the application. And more specifically — testing its integrations. share a branch with a colleague Keeping your running code isolated used to be much easier — you could run everything locally. I’ve had MySQL and Redis installed on almost every computer I’ve worked on. But as our applications grew to rely on managed services — like Cognito and BigQuery — running 100% locally became an issue. It’s possible to mock or mimic these services, but they never act exactly like their original counterparts. Furthermore — these integrations should be tested frequently and early — as the boundaries of our application tend to be the places we get surprised in, and . where things break Feature Environments to the rescue So if we want to test our application against real cloud services, why can’t we share? This actually works on a small scale. For example — each developer runs his code locally, and everyone works with a dedicated S3 bucket that has different folders to isolate usage. But you don’t have to think far to get to a point where this breaks. What if we’re using an SQS Queue? My local application would push a message into that queue, only to have a colleague’s local application pulling that message. That’s exactly the type of interruption we’d like to avoid. We should each use our own queue — and to extrapolate from there — . our own environment Running an environment for every feature branch, or every pull request, is firstly a matter of how you perceive your CD pipeline. Don’t think of it as ‘ ’, but as ‘ ’. You just need to give it a name. That could be ‘ ’ or ‘ ’, but it could also be ‘ ’ of ‘ ’. Your CD process needs to work for existing environments, as well as for new ones. This is where Terraform really shines. this deploys the environment this deploys an environment production staging pr1017 feature-blue-button Because Terraform uses a declarative approach, it is in charge of figuring out if the environment is new and needs to be created, or if it exists and needs to be updated. Another key feature of Terraform is , which isolate state between different environments (it was even called “environments” in previous versions). Your Terraform code will need to use the variable ${terraform.workspace} in order to make sure your resources are specific for the environment you are in. Workspaces A working example. We’ll work with a very simple setup, just to illustrate this flow. We’ll start with an S3 bucket, exposed as a website, containing a simple HTML file: locals { environment_name = terraform.workspace } “aws_s3_bucket” “website_bucket” { bucket = “ .feature.environment.blog.com” acl = “public-read” force_destroy = website { index_document = “index.html” } } resource ${local.environment_name} true Notice how we create a variable from the Terraform workspace name, and that we use that variable in order to give the bucket a unique-per-environment name. local Our “CD Process” will be a simple bash file, that accepts the environment name as a parameter: “Deploying environment ” “Injecting env vars” sed ‘s/!!!ENVIRONMENT!!!/’” ”’/g’ index.template.html > .index.html “Applying Terraform” terraform init “Selecting/creating workspace” terraform workspace select || terraform workspace new terraform apply echo $ENVIRONMENT echo $ENVIRONMENT $ENVIRONMENT echo echo $ENVIRONMENT $ENVIRONMENT Here we - Inject the environment name into an html file, which will serve as our static site. Select/create a Terraform workspace that’s named after our environment. Apply the environment. (You can find the complete example in https://github.com/env0/feature-environments-blog-code ) Let’s run it ! Running will deploy an environment called . Terraform will initialise, show us what actions are going to be performed (the Terraform plan), and after deploying everything, will output the endpoint of the website. If we go into that website — we’ll see something like this - ./apply.sh env1 env1 Let’s try running that again, for a different environment this time. We’ll run , and we’ll get a link to a website that looks like this — ./apply.sh my-other-env Updating In order to update our environment, all we have to do is run apply.sh again, with the same environment name. Our code, and Terraform, will recognize we are working on an existing environment, and will update it accordingly. Destroying When the branch is merged and deleted, there is no need for this environment, and we definitely don’t want to keep paying for it. In order to destroy that specific environment, we just need to run - > terraform workspace select > terraform destroy $ENVIRONMENT Terraform will ask for approval, and take it from there. Be careful though because those cat pics aren’t easy to find ;) Automating The next and final step to having feature-environments, is having them automated. This is an extremely beneficial thing to automate, and your engineers will thank you for getting a dedicated isolated environment for every branch/PR they open. You can take some version of the apply.sh file shown above and put it in your pipeline (GH actions, Circle, Jenkins, etc…). Just make sure you also remember the destroy part, when that branch/PR is closed. The joy of feature environments At , we’ve been using feature environments since day one. Every PR we open runs its own environment. That really helps us test our entire application early and without interruptions. We also commonly use this to showcase features in development, which really helps to get early feedback. It is also another incentive to open PR’s early in the development process, which is a great way to share what we are working on. Each developer runs about 20 environments every month, and the average environment lives for about 1.5 days. env0 From a blog post to the real world I would like to ask you to take all of the above with a grain of salt — this code is really an example, meant to illustrate a workflow. As software often goes — making it work “in the wild’’ is often much more complex. Automating Terraform and managing cloud environments isn’t a trivial thing, and most CI/CD tools are built to run one time tasks, and not manage “environments”, that have a longer lifecycle. oversimplified That’s why we created , a complete Environment-as-a-service solution, that is built to give your developers the dynamic infrastructure they need, without compromising on the organization’s needs to control and govern how cloud resources are used. Give it a go at . env0 www.env0.com Next steps - As I’ve written above, full blown feature environments are rarely as simple as the example above. So at this point, there are probably all kinds of questions you might be asking - This sounds expensive! What are some ways to make this cheaper? Giving everyone access to deploying infrastructure can get messy — how can I keep my cloud usage under control? I already have Terraform code — but it runs a static environment. What are the steps to making it dynamic? If you’d like to read more about any of the above — just let me know in the comments and I’ll try to write a follow up post. Thanks for reading! Previously published at https://medium.com/env0/from-feature-branches-to-feature-environments-with-terraform-652c0fdf0e78
