Introduction With containerization and container orchestration rapidly increasing, according to the , the usage of containers in production jumped 300% Cloud Native Survey from 2020 the usage of Kubernetes is only going to increase, but there is a difference between using Kubernetes and being able to extend it. Kubernetes is designed as a platform to build platforms and operators are the ultimate extensibility tools. The true power of Kubernetes is not just the ability to orchestrate containers, but rather its extensible API and . Kubernetes operators and control loops are how the internal machinery of everyone’s favourite container orchestrator works. Control Plane In this article, we will look at the operator pattern, learn when it is appropriate to use an operator. Finally, we will explore the operator’s architecture. Before you continue reading, please note that Kubernetes operators are difficult, take time to get right, and create maintenance challenges. However, if you find a good use case and can commit time and resources to master them, it will unlock additional powerful capabilities. Kubernetes Architecture Before we start looking into operators, let’s get a quick refresher on Kubernetes architecture: Internally Kubernetes architecture uses in closed control loops to ensure cluster health and correctness of the workloads by continuously reconciling the actual state of the cluster with the desired state specified by the user/administrator. controllers Some of the controllers on the diagram are: controller manager which manages all the Kubernetes native control loops cloud controller which enables cloud providers to plug in their infrastructure elements As you can see, control loops are at the core of how Kubernetes ensures state reconciliation and self-healing. Operators Overview The easiest way to understand operators is to think about them as software counterparts of human operators. Imagine someone performing tasks such as database backups, upgrades, schema changes, etc. There is a lot of complex operational knowledge involved to do all those tasks correctly. What if something fails? Well, even more knowledge and experience is needed to troubleshoot and recover from an indecent. Now imagine that you have complex software running on your cluster, something like , , or even . The knowledge required to properly operationalize all those components is encapsulated in operators. couchbase apache-spark Prometheus You might also find yourself on the provider end of the spectrum, and want to provide additional benefits to your customers. Writing and maintaining an operator is a fairly complex task, but if there is a use case, it’s well worth the effort. Kubernetes operators follow the . In simple terms, operator pattern automates operations performed typically by humans, such as: operator pattern installation updates and upgrades configuration backup gathering metrics troubleshooting Architecture Operators build on the Kubernetes architecture and all components are part of the Kubernetes standard building blocks. As you can see on the below-simplified diagram, the controller manager running on a control plane node is responsible for the reconciliation and management of all controllers, built-in and custom alike. A custom controller is just a reconciliation logic running as a program, written in any language, and deployed to a Kubernetes cluster. The simplest operator consists of: a custom controller running in a pod controlled by a Deployment or StatefulSet a CRD (Custom Resource Definition) defining schema for a custom resource a custom resource that forms an API layer that users can manipulate to interact with the operator The custom resource follows the same semantics as other Kubernetes resources. ![Operator Components Diagram ](https://cdn.hackernoon.com/images/Rd8z9ArVMycSceC003hxVrAXoj23-6603715.png) Maturity levels Operators can be as simple as installing components onto a cluster up to much more complex, like full life cycle management or even performance of configuration fine-tuning live as the application runs. There are 5 levels of operator maturity: Operators in practice Should you start writing operators for everything? Not, so when should you? Here are a few good use cases for operators: updating/upgrading stateful workloads interacting with external resources outside of a Kubernetes cluster chaos engineering advanced scheduling, configuration, and failure resiliency Writing and maintaining an operator is a very complex process, you need: deep understanding of Kubernetes internals expertise in any programming language (preferably Go) strong grasp on concepts like “eventual consistency”, “loose coupling”, “event-driven architecture” Before looking into creating your own operator, make sure to visit . There is a chance that the software you are using already has an operator. operator hub Controller vs operator What is the difference between controllers and operators? Well, there is no technical difference, but rather in semantics. Controller is a control loop continuously watching and reconciling state of the cluster. Operator is also a controller, but it additionally encapsulates specific domain knowledge. Helm vs operator If you look closer at what an operator is doing, it looks a lot like , at least from a packaging point of view. helm Whenever a question comes to “A vs B”, it is usually a good heuristics to rephrase it into “When is it more appropriate to use A and when B” Similarly here, helm is great for simple installation and templating, but the operator can do custom logic on the resources which helm simply was not designed for. Tooling There is a lot of operator frameworks, SDKs and related tooling, but I recommend looking at , also available in the form of an . kubebuilder online book Please refer to the last section of this blog “Resources” where you can find more information about available tools. Demo scenarios There are a few choices if you would like to experiment with operators. I recommend going through . kubebuilder’s quickstart use with a self-contained development environment if you have already a Kubernetes cluster and you would like to try something locally on your machine. Please note that this is a containerized environment, so other than Kubernetes and VS Code, you don’t need anything else. my repository follow an excellent by if you don’t want to install anything on your local machine and would rather prefer remote infrastructure. Katacoda scenario @javajon If you decided to use this repository, there are a few prerequisites: VS Code with Remote Development Extensions Docker Desktop with Kubernetes enabled. If you have other Kubernetes clusters in your config file, make sure to point to docker-desktop I’ve tested it with remote cluster and it works too, but doesn’t work with k3s/k3d. The image will take a while to load as it pulls kubebuilder, Go binaries, and other components, so please patient Summary For me, the main takeaway from learning about Kubernetes operators was a much deeper understanding of the Kubernetes architecture, but most importantly the realization that the true power of Kubernetes is in its extensible API and universal control plane. Whether it’s consuming operators or creating your own, starting with the mindset that Kubernetes is a platform to build platforms, is the path to levelling up your Kubernetes game. Resources Here are useful resources for further study. Kubernetes docs operator pattern CNCF Operators White Paper CNCF Blog controller for pod labels Video with hands-on operator building Build a Kubernetes operator RedHat Intro to operators RedHat Intro to operators IBM Git Awesome operator frameworks TGIK 040: Kubebuilder

Apache

SuperPower

YouTube

2022 - HackerNoon Contributor of the Year - Cloud Computing

2022 - HackerNoon Contributor of the Year - Devops

2022 - HackerNoon Contributor of the Year - Docker

2022 - HackerNoon Contributor of the Year - Github

2022 - HackerNoon Contributor of the Year - Kubernetes

2022 - HackerNoon Contributor of the Year - Open Source

Nominated for 2022 - HackerNoon Contributor of the Year - Open Source

Nominated for 2022 - HackerNoon Contributor of the Year - Github

Nominated for 2022 - HackerNoon Contributor of the Year - Docker

Nominated for 2022 - HackerNoon Contributor of the Year - Devops

Nominated for 2022 - HackerNoon Contributor of the Year - Cloud Computing

Nominated for 2022 - HackerNoon Contributor of the Year - Kubernetes

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

How To Get Better at Kubernetes

How To Get Better at Kubernetes

Too Long; Didn't Read

Companies Mentioned

Piotr

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

How To Get Better at Kubernetes

Too Long; Didn't Read

Companies Mentioned

Piotr

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES