: Did a on ufo, the slides are here: . Update presentation UFO Ship for AWS ECS Quick Introduction , ECS, is an service that provisions and manages containers on a cluster of EC2 instances. As with most of AWS services, it is great and simply requires a little tooling wrapped around it to create a smooth flow. is a simple tool that makes building and shipping Docker containers to AWS ECS super easy. Amazon EC2 Container Service AWS Docker Ufo Ufo provides a command called that does the following: ufo ship builds a docker image. generates and registers the ECS template definition. deploys the ECS template definition to the specified service. Ufo deploys a task definition that is created via a template generator which is . We’ll go over a quick example to show what the template looks like and how it works. fully controllable Task Definition ERB Template and DSL Generator The task definition is created from an ERB template in the folder. Here is an example: . ufo/templates ufo/templates/main.json.erb {"family": "<%= %>","containerDefinitions": [{"name": "<%= %>","image": "<%= %>","cpu": <%= %>,<% if %>"memory": <%= %>,<% end %><% if %>"memoryReservation": <%= %>,<% end %><% if %>"portMappings": [{"containerPort": "<%= %>","protocol": "tcp"}],<% end %>"command": <%= .to_json %>,<% if %>"environment": <%= .to_json %>,<% end %>"essential": true}]} @family @name @image @cpu @memory @memory @memory_reservation @memory_reservation @container_port @container_port @command @environment @environment The ERB template to use is specified in with the method. Ufo loads the ERB template when it evaluates the blocks in . ufo/task_definitions.rb source task_definition ufo/task_definition.rb task_definition "hi-web" dosource "main" # will use ufo/templates/main.json.erbvariables(family: task_definition_name,# image: tongueroo/hi:ufo-[timestamp]-[sha]image: helper.full_image_name,environment: helper.env_file('.env.prod')name: "web",container_port: helper.dockerfile_port,command: ["bin/web"])end As you can see above, the file has some special variables and helper methods available. These helper methods provide useful contextual information from the project so you don’t have to copy paste and update the code in multiple places. For example, one of the variable provides the exposed port in the Dockerfile of the project. Here is a list of the important ones: task_definitions.rb — The full docker image name that ufo builds. The “base” portion of the docker image name is defined in ufo/settings.yml. For example, the base portion is “tongueroo/hi” and the full image name is tongueroo/hi:ufo-[timestamp]-[sha]. So the base does not include the Docker tag and the full image name does include the tag. helper.full_image_name — Exposed port extracted from the Dockerfile of the project. helper.dockerfile_port — This method takes an .env file which contains a simple key value list of environment variables and converts the list to the proper task definition json format. helper.env_file The 2 classes which provide these special helper methods are in and . Refer to these classes for the full list of the special variables and methods. ufo/dsl.rb ufo/dsl/helper.rb Usage Example An example will demonstrate how easy it is to use ufo. More details are provided on the project’s official documentation: . http://ufoships.com This example is a demo rails app that returns the rails welcome page. I have also created and scripts that just run an infinite loop to mock out worker and clock processes for testing. The full source code for the demo project is available on GitHub: . bin/worker bin/clock tongueroo/hi Test Demo Project Locally in Mac OSX Let’s setup the app, install the dependencies and start up the web process. $ git clone $ cd hi$ bundle$ bin/web # start the web server https://github.com/tongueroo/hi Let’s curl for a 200 response code. $ curl -svo /dev/null localhost:3000 2>&1 | grep ‘< HTTP’< HTTP/1.1 200 OK$ Build and Test Demo Project with Docker Now let’s build the docker image with ufo and test that it works locally. I’m only showing some of the shell output to keep the paste small in size and useful in context. $ ufo init --app hi --image tongueroo/hiSetting up ufo project…created: ./bin/deployexists: ./Dockerfilecreated: ./ufo/settings.ymlcreated: ./ufo/task_definitions.rbcreated: ./ufo/templates/main.json.erbcreated: ./.envStarter ufo files created.$ ufo docker build$ docker images | grep “tongueroo/hi”tongueroo/hi ufo-2016–11–30T16–25–26-e1d57ce e511ec8a328a About a minute ago 826.8 MB$ docker run -d -p 3000:3000 — name hi tongueroo/hi:ufo-2016–11–30T16–25–26-e1d57ce$ docker ps$ curl -svo /dev/null localhost:3000 2>&1 | grep ‘< HTTP’< HTTP/1.1 200 OK$ docker stop hi ; docker rm hi$ In the above snippet, I ran two ufo commands: and . Let’s review some of the files that created: ufo init ufo docker build ufo init bin/deploy — Wrapper deploy script that shows how to deploy to 3 common web, worker and clock processes as services at the same time with ufo. ufo/settings.yml — This is where you set the Docker image name to be built. There is a service_cluster mapping option, which you can use to avoid having to always provide the option in the CLI. It is quite handy. --cluster ufo/task_definitions.rb — This is where you define the variables to be substituted into the ERB template. ufo/templates/main.json.erb — This the task definition ERB template. You can modify this and have full control over the task definition template that gets registered to ECS. .env — A starter .env file is provided if one does not yet exist on your project. You can rename this to and if that makes more sense for your needs. Remember to update the env_file line in if you rename the file. .env.prod .env.stag task_definitions.rb .env The command created a tongueroo/hi:ufo-2016–11–30T16–25–26-e1d57ce Docker image. ufo docker build Ship the Docker Image to ECS Let’s ship the web process as an ECS service. First, create an ECS Cluster called that we will use to ship the web service to. You will also need the ELB Target Group associated with the web service so that the app will be accessible from anywhere in the world. I have created a cluster, a “hi-elb” and “hi-target-group” for this example. You can grabbed the Target Group ARN from Load Balancing / Target Groups: stag stag You only need to provide the ELB Target Group ARN the first time deploying with ufo since you cannot update the Target Group of the ECS Service afterwards. If you want to change the Target Group, you must create and deploy a new service instead. This is an AWS ECS design decision. ufo ship Now we can ship the docker container to ECS with ! ufo ship $ ufo ship hi-web --cluster stag --target-group=arn:aws:elasticloadbalancing:us-east-1:467446852200:targetgroup/hi-target-group/f61e87b3c4761922Building docker image with:docker build -t tongueroo/hi:ufo-2016-12-01T07-37-53-e1d57ce -f Dockerfile .Docker image tongueroo/hi:ufo-2016-12-01T07-38-32-e1d57ce built. Took 2s.Pushed tongueroo/hi:ufo-2016-12-01T07-38-32-e1d57ce docker image. Took 4s.Building Task Definitions...Generating Task Definitions:Generated task definition at: ./ufo/output/hi-web.jsonGenerated task definition at: ./ufo/output/hi-worker.jsonGenerated task definition at: ./ufo/output/hi-clock.jsonTask Definitions built in ufo/output.hi-web task definition registered.Shipping hi-web...hi-web service created on stag clusterWaiting for deployment of task definition hi-web:3 to complete...........Time waiting for ECS deployment: 58s.Software shipped! Let’s inspect and review what the command actually did. First it built the docker image with a name of . You can check it out with . ufo ship tongueroo/hi:ufo-2016-12-01T07-37-53-ec1d57ce docker images $ docker images ufo-2016-12-01T07-38-32-e1d57ce a9e97fa264ab 5 minutes ago 826.8 MB$ Second, ufo generated all task definitions in the ufo/output folder and registers only the one that is deployed: hi-web. Let’s check out one of the generated task definitions: . ufo/output/hi-web.json {"family": "hi-web","containerDefinitions": [{"name": "web","image": "tongueroo/hi:ufo-2016-12-01T08-07-08-e1d57ce","cpu": 128,"memoryReservation": 256,"portMappings": [{"containerPort": "3000","protocol": "tcp"}],"command": ["bin/web"],"environment": [{"name": "ADMIN_PASSWORD","value": "secret"}],"essential": true}]} Third, ufo deploys the newly registered hi-web:3 task definition to ECS. Confirm It Works Grab the ELB DNS endpoint from EC2 Console / Load Balancing / Load Balancers. Confirm that the app is up with curl: $ curl -svo /dev/null hi-elb-1381308520.us-east-1.elb.amazonaws.com 2>&1 | grep '< HTTP'< HTTP/1.1 200 OK$ That’s it! The web process for this app has been deployed to ECS. tongueroo/hi Using bin/deploy When we use at the beginning of this example, it generated a script. This script handles deploying common application processes like web, worker and clock all at once. These processes typically use the same codebase, ie: same docker image, but have slightly different run time settings. For example, the docker run command for a web process could be and the command for a worker process could be . Environment variables are sometimes different also. ufo init bin/deploy puma sidekiq Let’s quickly test to make sure that the worker and clock process work locally first. The worker and clock process scripts are actually mocked out and are just simple infinite bash loops. That’s all we need to test things. Here’s a quick local test: $ bin/worker+ true+ echo 'fake worker process running...'fake worker process running...+ sleep 5^C$ bin/clock+ true+ echo 'fake clock process running...'fake clock process running...+ sleep 5^C$ Now let’s check out the script. bin/deploy #!/bin/bash -xe ufo ships hi-{web,clock,worker}-stag --cluster stag The bin/deploy wrapper script just calls the command, which is designed to build 1 docker image and ship it to multiple ECS services. Now let’s ship all 3 processes as services to ECS! ufo ships bin/deploy # deploys clock, worker and web! Summary The ufo tool automates building the docker image, registering the ECS task definition, and deploying the container to the ECS service. The project page is available on GitHub at . Try it out and let me know what you think! tongueroo/ufo Thanks for reading this far. If you found this post useful, I’d really appreciate it if you recommend this post (by clicking the clap button) so others can find it too! Also, connect with me on . LinkedIn
