Tekton Pipelines in the Azure Kubernetes Service Cluster

In this article, we will explore the installation of Azure Kubernetes Service (AKS) with the integration of Azure Container Instances as Virtual Nodes. On top of that, we will provision Tekton Pipelines and deploy a sample application to the on-demand Virtual Node.

Azure Kubernetes Service (AKS) is an Azure flavored managed Kubernetes Service. Kubernetes masters are managed by Azure and you only manage the worker nodes.

Azure Container Instances (ACI) is a Microsoft Azure managed service that allows you to schedule containers directly on the public cloud, without virtual machines (VMs). We will integrate it in the AKS cluster as a Virtual Node. Virtual nodes are often used for scaling solutions in AKS.

Virtual Nodes is a network communication between the AKS cluster and pods that run in Azure Container Instances (ACI). The virtual nodes add-on for AKS is based on the open-source project Virtual Kubelet.

Tekton Pipelines is a powerful Kubernetes-native open-source CI/CD project that allows you to test, build and deploy across cloud providers and on-premise data centers. The Ancient Greek noun tektōn (τέκτων) is a common term for an artisan/craftsman.

For this tutorial, we will use Azure Cloud Shell as a working environment. It is a nice free interactive shell that has common Azure tools preinstalled and configured to use with your account.

Prerequisites

• Azure subscription with contributor access

To use Azure Container Instances we need Microsoft.ContainerInstance service to be registered.

Check that Microsoft.ContainerInstance service provider is registered with your subscription, as shown in the following example:

az provider list --query "[?contains(namespace,'Microsoft.ContainerInstance')]" -o table

Microsoft.ContainerInstance should be in the Registered status. If not then go ahead and register it with the AZ CLI command:

az provider register --namespace Microsoft.ContainerInstance

Prepare dependency resources

Resource group

Now that prerequisites are sorted we will start by creating a resource group. Every resource in Azure should be in a particular resource group. In this example, I’m going to create a resource group tekton-lab in the westeurope region

az group create --name tekton-lab --location westeurope

Virtual Network and subnets

In this instruction, we will deploy Azure Kubernetes Cluster with advanced networking (Azure CNI) since virtual nodes will not work with AKS default basic networking (kubenet).

Let’s create a virtual network tekton-lab-Vnet and a subnet myAKSSubnet using the az network command:

az network vnet create \
    --resource-group tekton-lab \
    --name tekton-lab-Vnet \
    --address-prefixes 10.0.0.0/8 \
    --subnet-name myAKSSubnet \
    --subnet-prefix 10.240.0.0/16

Now we need to provision additional subnet for virtual nodes:

az network vnet subnet create \
    --resource-group tekton-lab \
    --vnet-name tekton-lab-Vnet \
    --name tekton-lab-VirtualNodeSubnet \
    --address-prefixes 10.241.0.0/16

Service principal

We will manually create a service principal and use it as our AKS cluster identity. This will allow it to interact with other Azure resources.

az ad sp create-for-rbac

expected output:

{
  "appId": "435be37-8310-f3d6-47af-e67c3184879",
  "displayName": "azure-cli-2022-03-05-17-32-55",
  "password": "Adkp1X38BLC31vmsEx7TV~k~bN.eLKT3T",
  "tenant": "1e2447db-4827-18fc--82f24e-84c3-0d00f"
}

Copy the appId and password. We will use those values in the next steps.

Grant permissions to AKS to use virtual network

We need to configure correct permissions for the AKS service principle to use network resources. To do that we need to obtain the resource ID of our network:

az network vnet show --resource-group tekton-lab --name tekton-lab-Vnet --query id -o tsv

Expected output:

/subscriptions/54a18e-db49-4300-afcc-01b049e24/resourceGroups/tekton-lab/providers/Microsoft.Network/virtualNetworks/tekton-lab-Vnet

Then we will create a role assignment with az role assignment create command to allow AKS cluster to use the virtual network:

az role assignment create --assignee 435be37-8310-f3d6-47af-e67c3184879 --scope /subscriptions/54a18e-db49-4300-afcc-01b049e24/resourceGroups/tekton-lab/providers/Microsoft.Network/virtualNetworks/tekton-lab-Vnet --role Contributor

Create AKS cluster

Get the ID of the subnet we created for the Azure Kubernetes Service cluster

az network vnet subnet show --resource-group tekton-lab --vnet-name tekton-lab-Vnet --name myAKSSubnet --query id -o tsv

Expected output:

/subscriptions/54a1468e-db49-4300-afcc-01f8b0493e24/resourceGroups/tekton-lab/providers/Microsoft.Network/virtualNetworks/tekton-lab-Vnet/subnets/myAKSSubnet

Now we will install AKS with az aks create command. It will take a few minutes to spin up the AKS cluster:

az aks create \
    --resource-group tekton-lab \
    --name myTektonAKSCluster \
    --node-count 1 \
    --node-vm-size standard_b2s \
    --generate-ssh-keys \
    --network-plugin azure \
    --service-cidr 10.0.0.0/16 \
    --dns-service-ip 10.0.0.10 \
    --docker-bridge-address 172.17.0.1/16 \
    --vnet-subnet-id /subscriptions/54a1468e-db49-4300-afcc-01f8b0493e24/resourceGroups/tekton-lab/providers/Microsoft.Network/virtualNetworks/tekton-lab-Vnet/subnets/myAKSSubnet \
    --service-principal 435be37-8310-f3d6-47af-e67c3184879 \
    --client-secret Adkp1X38BLC31vmsEx7TV~k~bN.eLKT3T

For client secret use <password> of service principle, we created in previous steps.

For service-principal use <appId> of service principle.

For vnet-subnet-id use the ID of the subnet in the previous step.

Enable addon for virtual nodes

We will use the az aks enable-addons command in order to enable virtual nodes and use the tekton-lab-VirtualNodeSubnet subnet that we created beforehand:

az aks enable-addons \
    --resource-group tekton-lab \
    --name myTektonAKSCluster \
    --addons virtual-node \
    --subnet-name tekton-lab-VirtualNodeSubnet

Get kubectl access

To get access to a newly created AKS cluster with kubectl run the following command:

az aks get-credentials --resource-group tekton-lab --name myTektonAKSCluster

Expected output: Merged "myTektonAKSCluster" as current context in /home/viachaslau/.kube/config

Check virtual node

To verify the connection to your cluster and the availability of the ACI virtual node, use the kubectl get command to return a list of the cluster nodes.

kubectl get nodes

You should be able to see your Azure Container Instances virtual nodes ready!

Tekton

Let’s install the latest version of Tekton with the following command:

kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

You can check the installation progress with the get pods command:

kubectl get pods --namespace tekton-pipelines

Now that Tekton is up and running let’s add Tekton Dashboard. It is a nice web-based UI for Tekton Pipelines.

kubectl apply --filename https://github.com/tektoncd/dashboard/releases/latest/download/tekton-dashboard-release.yaml

Tekton Dashboard

By default, Tekton Dashboard is accessible through its cluster IP type service. To quickly get web access from Cloud Shell we can set up port forwarding with Tekton Dashboard:

kubectl --namespace tekton-pipelines port-forward svc/tekton-dashboard 9097:9097

Get access to the Tekton Dashboard by clicking “Web preview” in Cloud Shell and then specify port 9097

Tekton task

Let’s create a sample Tekton task. TheTask is a series of Steps that you define in a particular order of execution as part of your pipeline. Task has inputs and could produce outputs. A Task executes as a Pod on your Kubernetes cluster.

Let’s create a tekton-task.yml file with the following content:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: hello
spec:
  steps:
    - name: hello
      image: ubuntu
      command:
        - echo
      args:
        - "Hello World!"

And then apply this file with kubectl apply command:

kubectl apply -f tekton-task.yml

If you navigate to the Tasks section in Tekton Dashboard you will see the “hello” task appeared:

Here is the YAML configuration of the “hello” task:

To run tasks specifically on the virtual node we need to get rid of its default taint. To get your node taints use the kubectl describe command:

kubectl describe node virtual-node-aci-linux

To remove taint you need to apply the same taint but with “-” character:

kubectl taint nodes virtual-node-aci-linux virtual-kubelet.io/provider=azure:NoSchedule-

Expected output: node/virtual-node-aci-linux untainted

To double-check that there is no taint left you can describe the node again:

Now that taints are sorted we can create a TaskRun for that task with node selector configured in UI. This way we will make sure that the Tekton task will run on Azure Container Instances.

Navigate to the TaskRuns section on the left menu and you will see that TaskRun “hello” succeded:

Summary: AKS Clusters with Azure Container Integration

In this tutorial, we created an AKS cluster with the integration of Azure Container Instances as Virtual Nodes with corresponding dependencies. We successfully installed Tekton Pipelines and Tekton Dashboard in the Azure Kubernetes Service cluster. Finally, we were able to schedule a sample Tekton task in an on-demand Virtual node.

Clean-up

Here is what to do to clean up resources created in this tutorial”

Delete Tekton installation

kubectl delete -f kubectl apply --filename https://github.com/tektoncd/dashboard/releases/latest/download/tekton-dashbo0
kubectl delete --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

To get rid of your virtual nodes, you can turn off them with az aks disable-addons command

az aks disable-addons --resource-group tekton-lab --name myTektonAKSCluster --addons virtual-node

Delete resource group with AKS:

az group delete --name tekton-lab --yes --no-wait