Never get a boring domain. Get .Tech!Download the 2020 State of API Report to Improve your API Strategy!
Visit SmartBear Software https://bit.ly/2Zn6y1d
promoted
Before you go, check out these stories!
0Creating A Scalable Application Using Docker + Rancher + RancherOs + Cattle by@eon01Creating A Scalable Application Using Docker + Rancher + RancherOs + Cattle
April 6th 2017 19,251 reads
@eon01Aymen El Amri
In the Docker World, orchestration is the most important part of the ecosystem. Docker Swarm, Kubernetes, Apache Mesos .. all of these are orchestrators, every one of them has its own philosophy, use cases, and architecture. Rancher is a tool built to simplify Docker orchestration and management.
Painless Docker is a practical guide to master Docker and its ecosystem based on real-world examples.
Painless Docker tends to be a complete and detailed guide to create, deploy, optimize, secure, trace, debug, log, orchestrate & monitor Docker and Docker clusters.
Through this online training, you will learn how to use Docker in development and production environments and the DevOps pipeline between them in order to build modern microservices applications.
This tutorial is part of this book and through it, we are going to discover how to use Rancher in order to create a scalable Wordpress application.
Rancher Architecture
In Rancher, everything (like containers, networks or images ) is an API resource with a process lifecycle. Containers, images, networks, and accounts are all API resources with their own process lifecycles.
Rancher is built on the top of containers:
- A web UI
- An API
- A server that manages Rancher Agents
- A database
- A machine microservice
- The docker-machine binary
When you run Rancher using docker run rancher/server ... the Rancher API + Rancher Process Server + The Database + Machine Microservice are processes that live inside this container.
Note that the docker-machine binary is also living in the same container but only runs when it is called by the API.
Rancher has also an Agent part that manages the life cycle of containers.
If docker-machine creates a machine successfully, some events are exchanged between the docker-machine and the microservice. A bootstrap event is created and a docker-machine configcommand is executed to get the details needed to connect to the machine’s Docker daemon.
If everything run without problems, the service fires up a Rancher Agent on the machine via docker
run rancher/agent …
Rancher Agents open a WebSocket connection to the server in order to establish a 2-way communication. The Rancher Agent manage its containers and reports every change using the Docker API.
During this tutorial, we are going to use an EC2 machine and this is how a different view of the layers of our Rancher installation:
RancherOS
RancherOS is a small distribution released by Rancher team. It is an easy way to run containers at scale in production and includes only the services needed to run Docker.
It only includes the latest version of Docker and removes any unneeded library that a “normal” Linux distribution could have.
In RancherOS, everything is a container, the traditional init system is replaced so that Docker run directly on the Kernel.
A special component in this system is called User Docker which is the daemon that allow a user (a non-system user) runs its containers.
We are going to run RancherOS in an EC2 machine using AWS CLI:
aws ec2 run-instances --image-id ami-ID --count 1 --instance-type t2.micro --key-name MySSHKeyName --security-groups sg-nameThis is the list of AMI by region:
Now you can login to your machine using the AWS common ssh command :
ssh -i "MySSHKeyName" rancher@xxxx.xx-xxxx-x.compute.amazonaws.comRunning Rancher
Since Docker is installed by default, we can start using it directly. Let’s start a MariaDB server then use it with Rancher Server.
docker run --name mariadb -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=cattle -e MYSQL_USER=cattle -e MYSQL_PASSWORD=cattle -p 3306:3306 -d mariadbRun the Rancher Server and change 172.31.0.190 by your MariaDB IP:
docker run --name rancher_server -p 8080:8080 -e CATTLE_DB_CATTLE_MYSQL_HOST=172.31.0.190 -e CATTLE_DB_CATTLE_MYSQL_PORT=3306 -e CATTLE_DB_CATTLE_MYSQL_NAME=cattle -e CATTLE_DB_CATTLE_USERNAME=cattle -e CATTLE_DB_CATTLE_PASSWORD=cattle -v /var/run/docker.sock:/var/run/docker.sock -d rancher/serverNow you can go to your <host_ip>:8080 and add a Linux host with a supported version of Docker:
We are not going to use the public IP address but the private (eth0) one.
Let’s add a custom host.
A command is given to run on any reachable host in order to let it join the server.
At this step, I already created an EC2 machines (with RancherOS as an operating system) and I am going to use it and run this command:
sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.1 http://172.31.0.190:8080/v1/scripts/02048221239BE78134BB:1483142400000:i5yrQRE4kFo7eIKz9n1o5VFTewMake sure any Security Groups or firewalls allow traffic from and to all other hosts on UDP ports 500 and 4500.
After the Agent installation, we can notice that some containers are running in each Agent machine like the dns, healthcheck, network-manager ..:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8339a8af3bb8 rancher/net:v0.9.4 "/rancher-entrypoint." About a minute ago Up About a minute r-ipsec-ipsec-router-1-19bd25e3
95f3d3afdb7c rancher/net:holder "/.r/r /rancher-entry" About a minute ago Up About a minute r-ipsec-ipsec-1-90844ad4
5643089ca485 rancher/healthcheck:v0.2.3 "/.r/r /rancher-entry" 2 minutes ago Up 2 minutes r-healthcheck-healthcheck-1-91ae658d
fec564c53f13 rancher/dns:v0.14.1 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-network-services-metadata-dns-1-471d9dea
c162c07d07f6 rancher/net:v0.9.4 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-ipsec-ipsec-cni-driver-1-e02f6a01
0313ff5cb812 rancher/scheduler:v0.7.5 "/.r/r /rancher-entry" 2 minutes ago Up 2 minutes r-scheduler-scheduler-1-5ca51856
0f6f62a190c7 rancher/network-manager:v0.5.3 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-network-services-network-manager-1-03d3f619
ec270864ff07 rancher/metadata:v0.8.11 "/rancher-entrypoint." 2 minutes ago Up 2 minutes r-network-services-metadata-1-4be6ebc6
98a78058bc3c rancher/agent:v1.2.1 "/run.sh run" 2 minutes ago Up 2 minutes rancher-agentRunning A Wordpress Service
If everything is ok, we are going to create a Wordpress service. Go to Stack menu and click User.
Create the MariaDB database, name it wordpressdb (we are going to use it in order to link it to the Wordpress container)
Then add your environment variables:
Now, using the same way, create the Wordpress container, map host port 80 to the container port 80 and link the mariadb container as a mysql instance.
Then add your environment variables:
At this step, we can check the running services, use Stack->User then default to see the containers:
What we started is the equivalent of 2 docker commands, one to start the mariadb container and the other one to start the wordpress container.
You can visit the Wordpress fresh installation using its IP address.
We can see the details of the different configurations a running container could have if you choose the container from the same view then use the “View in API” menu.
This is an example of the wordpress container:
{
"id":"1s7",
"type":"service",
"links":{
"self":"…/v2-beta/projects/1a5/services/1s7",
"account":"…/v2-beta/projects/1a5/services/1s7/account",
"consumedbyservices":"…/v2-beta/projects/1a5/services/1s7/consumedbyservices",
"consumedservices":"…/v2-beta/projects/1a5/services/1s7/consumedservices",
"instances":"…/v2-beta/projects/1a5/services/1s7/instances",
"networkDrivers":"…/v2-beta/projects/1a5/services/1s7/networkdrivers",
"serviceExposeMaps":"…/v2-beta/projects/1a5/services/1s7/serviceexposemaps",
"serviceLogs":"…/v2-beta/projects/1a5/services/1s7/servicelogs",
"stack":"…/v2-beta/projects/1a5/services/1s7/stack",
"storageDrivers":"…/v2-beta/projects/1a5/services/1s7/storagedrivers",
"containerStats":"…/v2-beta/projects/1a5/services/1s7/containerstats"
},
"actions":{
"upgrade":"…/v2-beta/projects/1a5/services/1s7/?action=upgrade",
"restart":"…/v2-beta/projects/1a5/services/1s7/?action=restart",
"update":"…/v2-beta/projects/1a5/services/1s7/?action=update",
"remove":"…/v2-beta/projects/1a5/services/1s7/?action=remove",
"deactivate":"…/v2-beta/projects/1a5/services/1s7/?action=deactivate",
"removeservicelink":"…/v2-beta/projects/1a5/services/1s7/?action=removeservicelink",
"addservicelink":"…/v2-beta/projects/1a5/services/1s7/?action=addservicelink",
"setservicelinks":"…/v2-beta/projects/1a5/services/1s7/?action=setservicelinks"
},
"baseType":"service",
"name":"wordpress",
"state":"active",
"accountId":"1a5",
"assignServiceIpAddress":false,
"createIndex":1,
"created":"2017-04-04T21:05:53Z",
"createdTS":1491339953000,
"currentScale":1,
"description":"wordpress files",
"externalId":null,
"fqdn":null,
"healthState":"healthy",
"instanceIds":[
"1i14"
],
"kind":"service",
"launchConfig":{
"type":"launchConfig",
"capAdd":[],
"capDrop":[
],
"dataVolumes":[
],
"dataVolumesFrom":[
],
"devices":[
],
"dns":[
],
"dnsSearch":[
],
"environment":{
"WORDPRESS_DB_NAME":"wordpress",
"WORDPRESS_DB_USER":"wordpress",
"WORDPRESS_DB_PASSWORD":"wordpress"
},
"imageUuid":"docker:wordpress",
"instanceTriggeredStop":"stop",
"kind":"container",
"labels":{
"io.rancher.container.pull_image":"always"
},
"logConfig":{
"type":"logConfig",
"config":{
},
"driver":""
},
"networkMode":"managed",
"ports":[
"80:80/tcp"
],
"privileged":false,
"publishAllPorts":false,
"readOnly":false,
"secrets":[
],
"startOnCreate":true,
"stdinOpen":true,
"system":false,
"tty":true,
"version":"0",
"dataVolumesFromLaunchConfigs":[
],
"vcpu":1
},
"lbConfig":null,
"linkedServices":{
"mysql":"1s6"
},
"metadata":null,
"publicEndpoints":[
{
"type":"publicEndpoint",
"hostId":"1h1",
"instanceId":"1i14",
"ipAddress":"54.246.163.197",
"port":80,
"serviceId":"1s7"
}
],
"removed":null,
"retainIp":null,
"scale":1,
"scalePolicy":null,
"secondaryLaunchConfigs":[
],
"selectorContainer":null,
"selectorLink":null,
"stackId":"1st5",
"startOnCreate":true,
"system":false,
"transitioning":"no",
"transitioningMessage":null,
"transitioningProgress":null,
"upgrade":null,
"uuid":"0b8a8290-0d10-47be-b182-45f1e57e80bc",
"vip":null
}
We can also inspect other services and hosts configurations in the same way.
Cattle: The Rancher Container Orchestrator
What we started below is Rancher powered by its own orchestration tool called Cattle. Rancher offers the possibility to use other orchestration tools like Kubernetes, Docker Swarm or Mesos.
Cattle is a container orchestration and scheduling framework, in its beginning, it was designed as an extension to Docker Swarm but since Docker Swarm continues to develop, Cattle and Swarm started to diverge. Cattle is used extensively by Rancher itself to orchestrate infrastructure services as well as setting up, managing, and upgrading Swarm, Kubernetes, and Mesos clusters.
Cattle application deployments are organized into stacks which can be used to User or Infrastructure stacks.
A Cattle Stack is a collection of Services and the latter is primarily a Docker image with its networking, scalability, storage, health checks, service discovery links, environment and all of the other configurations.
A Cattle Service could be a load balancer or an external service. A Stack can be launched using a docker-compose.yml or rancher-compose.yml file or just start containers like we did for the Wordpress stack.
In addition to this, you can start several applications using an application catalog. More than 50 different apps are available in the app catalog.
An application is defined by a docker-compose and a rancher-compose file and can be deployed easily with the default configurations.
Let’s take the example of Portainer:
Portainer is a lightweight management UI which allows you to easily manage your Docker host or Swarm cluster.
Portainer is meant to be as simple to deploy as it is to use. It consists of a single container that can run on any Docker engine (Docker for Linux and Docker for Windows are supported).
Portainer allows you to manage your Docker containers, images, volumes, networks and more ! It is compatible with the standalone Docker engine and with Docker Swarm.
This is the docker-compose.yml file of Portainer:
portainer:
labels:
io.rancher.sidekicks: ui
io.rancher.container.create_agent: true
io.rancher.container.agent.role: environment
image: rancher/portainer-agent:v0.1.0
volumes:
- /configui:
image: portainer/portainer:pr572
command: --no-auth --external-endpoints=/config/config.json --sync-interval=5s -p :80
volumes_from:
- portainer
net: container:portainerThis is its rancher-compose.yml file:
.catalog:
name: "Portainer"
version: "1.11.4"
description: Open-source lightweight management UI for a Docker host or Swarm cluster
minimum_rancher_version: v1.5.0-rc1
Rancher offers the possibility to start infrastructure Stacks. If you go to Stacks->Infrastructure, you can see a catalog of tools like Bind9, Portainer, Rancher vxlan ..etc
Scaling Wordpress Using Rancher
Now we want to scale our Wordpress frontal app in order to handle more traffic. In this case, you should click Stack->User then the name of the Stack (it should be the default one if you followed this tutorial as it is). Now click on Wordpress and scale your app to 3 containers:
At this step, you should see how Wordpress won’t scale the right way.
This is normal since all of our Wordpress containers are mapped to the same port 80 (that can handle only one container).
In order to scale Wordpress the right way, we should tell Rancher to create a load balancer service with auto-discovery and let Rancher choose the Wordpress container port automatically.
This part is detailed in Painless Docker Book.
Connect Deeper
This article is part of Painless Docker Book: Unlock The Power Of Docker & Its Ecosystem.
Painless Docker is a practical guide to master Docker and its ecosystem based on real-world examples.
Painless Docker tends to be a complete and detailed guide to create, deploy, optimize, secure, trace, debug, log, orchestrate & monitor Docker and Docker clusters. Through this book, you will learn how to use Docker in development and production environments and the DevOps pipeline between them in order to build modern microservices applications.
If you resonated with this article, please subscribe to DevOpsLinks : An Online Community Of Diverse & Passionate DevOps, SysAdmins & Developers From All Over The World.
You can find me on Twitter and you can also check my books: SaltStack For DevOps & Painless Docker.
If you liked this post, please recommend and share it with your followers.
Related
Tags
Subscribe to get your daily round-up of top tech stories!