Creating A Scalable Application Using Docker + Rancher + RancherOs + Cattle

This tutorial is part of this book and through it, we are going to discover how to use Rancher in order to create a scalable application. Wordpress Rancher Architecture In , everything (like containers, networks or images ) is an resource with a process lifecycle. Containers, images, networks, and accounts are all resources with their own process lifecycles. Rancher API API Rancher is built on the top of containers: A web UI An API A server that manages Rancher Agents A database A machine microservice The binary docker-machine Rancher Server Architecture When you run using the + + The Database + are processes that live inside this container. Rancher docker run rancher/server ... Rancher API Rancher Process Server Machine Microservice Note that the docker-machine binary is also living in the same container but only runs when it is called by the . API has also an Agent part that manages the life cycle of containers. Rancher If creates a machine successfully, some events are exchanged between the and the microservice. A event is created and a command is executed to get the details needed to connect to the machine’s Docker daemon. docker-machine docker-machine bootstrap docker-machine config If everything run without problems, the service fires up a on the machine via docker Rancher Agent run rancher/agent … open a connection to the server in order to establish a 2-way communication. The manage its containers and reports every change using the Docker . Rancher Agents WebSocket Rancher Agent API During this tutorial, we are going to use an machine and this is how a different view of the layers of our installation: EC2 Rancher RancherOS Architecture RancherOS is a small distribution released by team. It is an easy way to run containers at scale in production and includes only the services needed to run Docker. RancherOS Rancher It only includes the latest version of Docker and removes any unneeded library that a “normal” distribution could have. Linux In , everything is a container, the traditional system is replaced so that Docker run directly on the . RancherOS init Kernel A special component in this system is called which is the daemon that allow a user (a non-system user) runs its containers. User Docker RancherOS Architecture We are going to run in an machine using : RancherOS EC2 AWS CLI aws ec2 run-instances --image-id ami-ID --count 1 --instance-type t2.micro --key-name MySSHKeyName --security-groups sg-name This is the list of AMI by region: Now you can login to your machine using the common command : AWS ssh ssh -i "MySSHKeyName" rancher@xxxx.xx-xxxx-x.compute.amazonaws.com Running Rancher Since Docker is installed by default, we can start using it directly. Let’s start a server then use it with . MariaDB Rancher Server docker run --name mariadb -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=cattle -e MYSQL_USER=cattle -e MYSQL_PASSWORD=cattle -p 3306:3306 -d mariadb Run the and change 172.31.0.190 by your : Rancher Server MariaDB IP docker run --name rancher_server -p 8080:8080 -e CATTLE_DB_CATTLE_MYSQL_HOST=172.31.0.190 -e CATTLE_DB_CATTLE_MYSQL_PORT=3306 -e CATTLE_DB_CATTLE_MYSQL_NAME=cattle -e CATTLE_DB_CATTLE_USERNAME=cattle -e CATTLE_DB_CATTLE_PASSWORD=cattle -v /var/run/docker.sock:/var/run/docker.sock -d rancher/server Now you can go to your and add a host with a supported version of Docker: :8080 Linux Adding A Rancher Host We are not going to use the public address but the private ( ) one. IP eth0 Adding A Rancher Host Let’s add a custom host. A command is given to run on any reachable host in order to let it join the server.At this step, I already created an machines (with as an operating system) and I am going to use it and run this command: EC2 RancherOS sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.1 http://172.31.0.190:8080/v1/scripts/02048221239BE78134BB:1483142400000:i5yrQRE4kFo7eIKz9n1o5VFTew Make sure any or allow traffic from and to all other hosts on ports 500 and 4500. Security Groups firewalls UDP After the installation, we can notice that some containers are running in each machine like the , , ..: Agent Agent dns healthcheck network-manager CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES8339a8af3bb8 rancher/net:v0.9.4 "/rancher-entrypoint." About a minute ago Up About a minute r-ipsec-ipsec-router-1-19bd25e395f3d3afdb7c rancher/net:holder "/.r/r /rancher-entry" About a minute ago Up About a minute r-ipsec-ipsec-1-90844ad45643089ca485 rancher/healthcheck:v0.2.3 "/.r/r /rancher-entry" 2 minutes ago Up 2 minutes r-healthcheck-healthcheck-1-91ae658dfec564c53f13 rancher/dns:v0.14.1 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-network-services-metadata-dns-1-471d9deac162c07d07f6 rancher/net:v0.9.4 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-ipsec-ipsec-cni-driver-1-e02f6a010313ff5cb812 rancher/scheduler:v0.7.5 "/.r/r /rancher-entry" 2 minutes ago Up 2 minutes r-scheduler-scheduler-1-5ca518560f6f62a190c7 rancher/network-manager:v0.5.3 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-network-services-network-manager-1-03d3f619ec270864ff07 rancher/metadata:v0.8.11 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-network-services-metadata-1-4be6ebc698a78058bc3c rancher/agent:v1.2.1 "/run.sh run" 2 minutes ago Up 2 minutes rancher-agent Running A Wordpress Service If everything is ok, we are going to create a service. Go to menu and click . Wordpress Stack User Create the database, name it (we are going to use it in order to link it to the container) MariaDB wordpressdb Wordpress Then add your environment variables: Now, using the same way, create the container, map host port 80 to the container port 80 and link the container as a instance. Wordpress mariadb mysql Then add your environment variables: At this step, we can check the running services, use -> then to see the containers: Stack User default What we started is the equivalent of 2 docker commands, one to start the container and the other one to start the container. mariadb wordpress You can visit the fresh installation using its address. Wordpress IP We can see the details of the different configurations a running container could have if you choose the container from the same view then use the “ ” menu. View in API This is an example of the container: wordpress {"id":"1s7","type":"service","links":{"self":"…/v2-beta/projects/1a5/services/1s7","account":"…/v2-beta/projects/1a5/services/1s7/account","consumedbyservices":"…/v2-beta/projects/1a5/services/1s7/consumedbyservices","consumedservices":"…/v2-beta/projects/1a5/services/1s7/consumedservices","instances":"…/v2-beta/projects/1a5/services/1s7/instances","networkDrivers":"…/v2-beta/projects/1a5/services/1s7/networkdrivers","serviceExposeMaps":"…/v2-beta/projects/1a5/services/1s7/serviceexposemaps","serviceLogs":"…/v2-beta/projects/1a5/services/1s7/servicelogs","stack":"…/v2-beta/projects/1a5/services/1s7/stack","storageDrivers":"…/v2-beta/projects/1a5/services/1s7/storagedrivers","containerStats":"…/v2-beta/projects/1a5/services/1s7/containerstats"},"actions":{"upgrade":"…/v2-beta/projects/1a5/services/1s7/?action=upgrade","restart":"…/v2-beta/projects/1a5/services/1s7/?action=restart","update":"…/v2-beta/projects/1a5/services/1s7/?action=update","remove":"…/v2-beta/projects/1a5/services/1s7/?action=remove","deactivate":"…/v2-beta/projects/1a5/services/1s7/?action=deactivate","removeservicelink":"…/v2-beta/projects/1a5/services/1s7/?action=removeservicelink","addservicelink":"…/v2-beta/projects/1a5/services/1s7/?action=addservicelink","setservicelinks":"…/v2-beta/projects/1a5/services/1s7/?action=setservicelinks"},"baseType":"service","name":"wordpress","state":"active","accountId":"1a5","assignServiceIpAddress":false,"createIndex":1,"created":"2017-04-04T21:05:53Z","createdTS":1491339953000,"currentScale":1,"description":"wordpress files","externalId":null,"fqdn":null,"healthState":"healthy","instanceIds":["1i14"],"kind":"service","launchConfig":{"type":"launchConfig","capAdd":[ ],"capDrop":[ ],"dataVolumes":[ ],"dataVolumesFrom":[ ],"devices":[ ],"dns":[ ],"dnsSearch":[ ],"environment":{"WORDPRESS_DB_NAME":"wordpress","WORDPRESS_DB_USER":"wordpress","WORDPRESS_DB_PASSWORD":"wordpress"},"imageUuid":"docker:wordpress","instanceTriggeredStop":"stop","kind":"container","labels":{"io.rancher.container.pull_image":"always"},"logConfig":{"type":"logConfig","config":{ },"driver":""},"networkMode":"managed","ports":["80:80/tcp"],"privileged":false,"publishAllPorts":false,"readOnly":false,"secrets":[ ],"startOnCreate":true,"stdinOpen":true,"system":false,"tty":true,"version":"0","dataVolumesFromLaunchConfigs":[ ],"vcpu":1},"lbConfig":null,"linkedServices":{"mysql":"1s6"},"metadata":null,"publicEndpoints":[{"type":"publicEndpoint","hostId":"1h1","instanceId":"1i14","ipAddress":"54.246.163.197","port":80,"serviceId":"1s7"}],"removed":null,"retainIp":null,"scale":1,"scalePolicy":null,"secondaryLaunchConfigs":[ ],"selectorContainer":null,"selectorLink":null,"stackId":"1st5","startOnCreate":true,"system":false,"transitioning":"no","transitioningMessage":null,"transitioningProgress":null,"upgrade":null,"uuid":"0b8a8290-0d10-47be-b182-45f1e57e80bc","vip":null} We can also inspect other services and hosts configurations in the same way. Cattle: The Rancher Container Orchestrator What we started below is powered by its own orchestration tool called . Rancher offers the possibility to use other orchestration tools like , or . Rancher Cattle Kubernetes Docker Swarm Mesos is a container orchestration and scheduling framework, in its beginning, it was designed as an extension to but since continues to develop, and started to diverge. is used extensively by itself to orchestrate infrastructure services as well as setting up, managing, and upgrading , , and clusters. Cattle Docker Swarm Docker Swarm Cattle Swarm Cattle Rancher Swarm Kubernetes Mesos application deployments are organized into which can be used to or stacks. A is a collection of and the latter is primarily a Docker image with its networking, scalability, storage, health checks, service discovery links, environment and all of the other configurations. Cattle stacks User Infrastructure Cattle Stack Services A could be a load balancer or an external service. A can be launched using a or file or just start containers like we did for the stack. Cattle Service Stack docker-compose.yml rancher-compose.yml Wordpress In addition to this, you can start several applications using an application catalog. More than 50 different apps are available in the app catalog. An application is defined by a and a file and can be deployed easily with the default configurations. docker-compose rancher-compose Let’s take the example of : Portainer Portainer is a lightweight management UI which allows you to easily manage your Docker host or Swarm cluster.Portainer is meant to be as simple to deploy as it is to use. It consists of a single container that can run on any Docker engine (Docker for Linux and Docker for Windows are supported).Portainer allows you to manage your Docker containers, images, volumes, networks and more ! It is compatible with the standalone Docker engine and with Docker Swarm. This is the file of : docker-compose.yml Portainer portainer: labels: io.rancher.sidekicks: ui io.rancher.container.create_agent: true io.rancher.container.agent.role: environment image: rancher/portainer-agent:v0.1.0 volumes: - /config ui: image: portainer/portainer:pr572 command: --no-auth --external-endpoints=/config/config.json --sync-interval=5s -p :80 volumes_from: - portainer net: container:portainer This is its file: rancher-compose.yml .catalog: name: "Portainer" version: "1.11.4" description: Open-source lightweight management UI for a Docker host or Swarm cluster minimum_rancher_version: v1.5.0-rc1 Starting Portainer Rancher offers the possibility to start infrastructure . If you go to -> , you can see a catalog of tools like , , ..etc Stacks Stacks Infrastructure Bind9 Portainer Rancher vxlan Infrastructure Catalog Scaling Wordpress Using Rancher Now we want to scale our frontal app in order to handle more traffic. In this case, you should click -> then the name of the (it should be the default one if you followed this tutorial as it is). Now click on and scale your app to 3 containers: Wordpress Stack User Stack Wordpress At this step, you should see how won’t scale the right way. Wordpress This is normal since all of our containers are mapped to the same port 80 (that can handle only one container). Wordpress In order to scale the right way, we should tell Rancher to create a load balancer service with auto-discovery and let choose the container port automatically. Wordpress Rancher Wordpress This part is detailed in . Painless Docker Book Connect Deeper This article is part of . Painless Docker Book: Unlock The Power Of Docker & Its Ecosystem is a practical guide to master and based . Painless Docker Docker its ecosystem on real-world examples Painless Docker tends to be a complete and detailed guide to create, deploy, optimize, secure, trace, debug, log, orchestrate & monitor Docker and Docker clusters. Through this book, you will learn how to use Docker in development and production environments and the DevOps pipeline between them in order to build modern microservices applications. If you resonated with this article, please subscribe to : An Online Community Of Diverse & Passionate DevOps, SysAdmins & Developers From All Over The World. DevOpsLinks You can find me on and you can also check my books: & . Twitter SaltStack For DevOps Painless Docker If you liked this post, please recommend and share it with your followers.