My goal for today is installing a simple cluster on Azure with: Hybrid Kubernetes Microsoft 1 Linux Master 2 Linux Nodes in a LinuxAgentPool 2 Windows Nodes in a WindowsAgentPool This is something that can’t be accomplished out of the box with the current version of Azure Container Services, but yesterday confirmed they are working on it! Brendan Burns So how can we install this cluster right now? The answer is an Open Source project from Microsoft that takes a “cluster definition” in input and returns an ARM template with additional artifacts (i.e. kubectl export files) as output. ACS-Engine Let’s start! To make thing easier, I’ll reuse some of the artifacts created in and . Part 1 Part 2 Create a folder named to store all your working artifacts acs Clone ACS Engine Repository: git clone [https://github.com/Azure/acs-engine](https://github.com/Azure/acs-engine) Go in the Example folder: where you can find many cluster definition examples. Here you can see a folder, it contains the definition needed: cd acs-engine/examples windows kubernetes-hybrid.json : There is an with the current that leads to examples in source code not being compatible with it (they are more updated). Here we have 2 solutions: Important note issue release 0.5.0 : Compile a new Release locally Complex : Change Examples to make them compatible with old version Easy We will choose the complex one! Joking… I followed the complex but it’s not trivial on macOS at the moment, because you need to clone acs-engine source in $GOPATH/src and needs a separate article on its own. So let’s go for the easy solution. Open file in your favorite editor and change lines below kubernets-hybrid.json from: "servicePrincipalProfile": {"clientId": "","secret": ""} to: "servicePrincipalProfile": {" ": ""," ": ""} servicePrincipalClientID servicePrincipalClientSecret Wait don’t close the file! We want to use latest Kubernetes version in our cluster right? So add line in bold: "orchestratorProfile": {"orchestratorType": "Kubernetes", }, "orchestratorVersion": "1.7.2" Now we can follow steps in that describes a and way to deploy your cluster. Deploy a Kubernetes Cluster short long Short way drawbacks Here we will follow the short way but before starting I want to highlight some drawbacks here: DNS prefix: a random number will be added to prefix passed as arguments to make it unique A new application (ServicePrincipal) will be created on you subscription each time you run deploy command and will be left there when you delete ResourceGroup because it’s not part of it A new SSH key pair will be created for deploy : We can avoid all these drawbacks creating artifacts once (application, SSH key pair, DNS) and adding their value in template json file used for deployment. I will do this for SSH. Solution 1 : At the end of the first deploy, acs-engine generates a _output folder with generated artifacts. Here there is a file called that contains all values for parameters that can be copied in the cluster definition file used. That’s all. Solution 2 azuredeploy.parameters.json Deployment In previous parts of this article, I created an SSH key pair and I want to use instead of generating a new one. This can be accomplished copying the content of the .pub file in the part below of the json file. Next steps will skip ssh key pair generation because already found in template. "linuxProfile": {"adminUsername": "azureuser","ssh": {"publicKeys": [{"keyData": "YOURKEY starts with ssh-rsa"}]}}, For deploy you need subscription id you are working on that can be easily retrieved with az account show ./acs-engine deploy — subscription-id YOURSUBSCRIPTIONIDHERE — dns-prefix ivank8stest — location westeurope — auto-suffix — api-model ~/acs/acs-engine/examples/windows/kubernetes-hybrid.json : If you want to avoid passing dns-prefix as argument you can add it to template.json file as we did for same trick can be used for any elements in the file. acs-engine is smart enough to skip steps if data are present. i.e. ServicePrincipalProfile creation. Note keyData If everything went well you should see something like: WARN[0002] apimodel: missing masterProfile.dnsPrefix will use “ivank8stest-59a13555”WARN[0002] apimodel: ServicePrincipalProfile was missing or empty, creating application…WARN[0004] created application with applicationID (11111111-048e-420f-afad-1fe450036077) and servicePrincipalObjectID (22222222-b0fb-4395–9576–6bb18481f88f).WARN[0004] apimodel: ServicePrincipalProfile was empty, assigning role to application…INFO[0030] Starting ARM Deployment (myAcsTest-1222046555). This will take some time… Coffee break is needed! INFO[0526] Finished ARM Deployment (myAcsTest2–676730522). We did it! Our Hybrid Kubernetes Cluster seems up and running! Let’s test it now. Connect to your Kubernetes cluster When deployment is completed you should have a folder with a subfolder named as dns-prefix of your super cluster. Go in it. _output It contains artifacts generated by acs-engine and is really useful for us as described in step 9. I will try to make it super simple here. kubeconfig here From your terminal run following command point to your _output folder export KUBECONFIG=~/acs/acs-engine/_output/ivank8stest-59a13ac6/kubeconfig/kubeconfig.westeurope.jsonkubectl get nodes NAME                        STATUS                        AGE       VERSION23586acs9010                Ready                         1s        v1.7.2-4+b0c9ea2463aba423586acs9011                Ready                         3s        v1.7.2-4+b0c9ea2463aba4k8s-linuxpool1-23586643-0   NotReady                      2s        v1.7.2k8s-linuxpool1-23586643-1   NotReady                      5s        v1.7.2k8s-master-23586643-0       NotReady,SchedulingDisabled   7s        v1.7.2 It’s up and running and we can use it with our local kubectl! Everything is up and running, let’s try to deploy some Linux and Windows containers! Linux Containers We are going to deploy same azure vote app deployed in , but with a small change in the yaml file needed to deploy on os:linux nodes. Part 1 Here is the content of the azure-vote.yaml file with changes highlighted: apiVersion: apps/v1beta1kind: Deploymentmetadata:name: azure-vote-backspec:replicas: 1template:metadata:labels:app: azure-vote-backspec:containers:- name: azure-vote-backimage: redisports:— containerPort: 6379name: redis ---apiVersion: v1kind: Servicemetadata:name: azure-vote-backspec:ports:— port: 6379selector:app: azure-vote-back---apiVersion: apps/v1beta1kind: Deploymentmetadata:name: azure-vote-frontspec:replicas: 1template:metadata:labels:app: azure-vote-frontspec:containers:— name: azure-vote-frontimage: microsoft/azure-vote-front:redis-v1ports:— containerPort: 80env:— name: REDISvalue: "azure-vote-back" ---apiVersion: v1kind: Servicemetadata:name: azure-vote-frontspec:type: LoadBalancerports:— port: 80selector:app: azure-vote-front nodeSelector:beta.kubernetes.io/os: linux nodeSelector:beta.kubernetes.io/os: linux You can now deploy it using: kubectl create -f azure-vote.yaml Leave Kubernetes working and let’s move to… Windows Containers Here we will use artifacts from but condensed in a single iisdeploymentfull.yaml file describing the whole deployment. Part 2 apiVersion: apps/v1beta1kind: Deploymentmetadata:name: iisspecreplicas: 2template:metadata:labels:app: iisspec:containers:- name: iisimage: microsoft/iisports:- containerPort: 80name: iisnodeSelector:beta.kubernetes.io/os: windows---apiVersion: v1kind: Servicemetadata:name: iisspec:ports: port: 80selector:app: iis Again we can deploy it using: kubectl create -f iisdeploymentfull.yaml Now we need to wait few minutes to have everything up and running. Check status from Kubernetes Dashboard Connect to your Kubernetes Dashboard through usual proxy channel: and open a browser: kubectl proxy http://127.0.0.1:8001/ui From here feel free to navigate in the various sections to check status of your cluster, pods, deployments and services. From Services you will be able to retrive External Endpoints that you can use to test both iis and azure-vote deployments. Let’s close this article with a quick scale of both deployments to push our cluster: kubectl scale deployments/azure-vote-front --replicas 100 kubectl scale deployments/iis --replicas 4 Final result: Kubernetes Hybrid Cluster running on Microsoft Azure As always you can delete everything with a simple Azure CLI 2 command: az group delete --name myAcsTest --yes --no-wait We did it again! Now we are Azure Kubernetes Masters and we can easily deploy whatever we want! My adventure will continue with ingress distributed on Linux and Windows nodes, Monitoring, Autoscale on Azure and much more.

Microsoft

Super

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

Kubernetes Adventures on Azure — Part 3 (ACS Engine & Hybrid Cluster)

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Ivan Fioravanti

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

Kubernetes Adventures on Azure — Part 3 (ACS Engine & Hybrid Cluster)

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Ivan Fioravanti

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES