With containerization and container orchestration rapidly increasing,
according to the Cloud Native Survey from 2020, the usage of containers in production jumped 300%
the usage of Kubernetes is only going to increase, but there is a difference between using Kubernetes and being able to extend it.
Kubernetes is designed as a platform to build platforms and operators are the ultimate extensibility tools.
The true power of Kubernetes is not just the ability to orchestrate containers, but rather its extensible API and Control Plane. Kubernetes operators and control loops are how the internal machinery of everyone’s favourite container orchestrator works.
In this article, we will look at the operator pattern, learn when it is appropriate to use an operator. Finally, we will explore the operator’s architecture.
Before you continue reading, please note that Kubernetes operators are difficult, take time to get right, and create maintenance challenges. However, if you find a good use case and can commit time and resources to master them, it will unlock additional powerful capabilities.
Before we start looking into operators, let’s get a quick refresher on Kubernetes architecture:
Internally Kubernetes architecture uses controllers in closed control loops to ensure cluster health and correctness of the workloads by continuously reconciling the actual state of the cluster with the desired state specified by the user/administrator.
Some of the controllers on the diagram are:
controller manager which manages all the Kubernetes native control loops
cloud controller which enables cloud providers to plug in their infrastructure elements
As you can see, control loops are at the core of how Kubernetes ensures state reconciliation and self-healing.
The easiest way to understand operators is to think about them as software counterparts of human operators. Imagine someone performing tasks such as database backups, upgrades, schema changes, etc. There is a lot of complex operational knowledge involved to do all those tasks correctly. What if something fails? Well, even more knowledge and experience is needed to troubleshoot and recover from an indecent.
Now imagine that you have complex software running on your cluster, something like couchbase, apache-spark, or even Prometheus. The knowledge required to properly operationalize all those components is encapsulated in operators.
You might also find yourself on the provider end of the spectrum, and want to provide additional benefits to your customers. Writing and maintaining an operator is a fairly complex task, but if there is a use case, it’s well worth the effort.
Kubernetes operators follow the operator pattern. In simple terms, operator pattern automates operations performed typically by humans, such as:
Operators build on the Kubernetes architecture and all components are part of the Kubernetes standard building blocks. As you can see on the below-simplified diagram, the controller manager running on a control plane node is responsible for the reconciliation and management of all controllers, built-in and custom alike. A custom controller is just a reconciliation logic running as a program, written in any language, and deployed to a Kubernetes cluster.
The simplest operator consists of:
The custom resource follows the same semantics as other Kubernetes resources.
![Operator Components Diagram
Operators can be as simple as installing components onto a cluster up to much more complex, like full life cycle management or even performance of configuration fine-tuning live as the application runs.
There are 5 levels of operator maturity:
Should you start writing operators for everything? Not, so when should you? Here are a few good use cases for operators:
Writing and maintaining an operator is a very complex process, you need:
Before looking into creating your own operator, make sure to visit operator hub. There is a chance that the software you are using already has an operator.
What is the difference between controllers and operators? Well, there is no technical difference, but rather in semantics.
Controller is a control loop continuously watching and reconciling state of the cluster. Operator is also a controller, but it additionally encapsulates specific domain knowledge.
If you look closer at what an operator is doing, it looks a lot like helm, at least from a packaging point of view.
Whenever a question comes to “A vs B”, it is usually a good heuristics to rephrase it into “When is it more appropriate to use A and when B”
Similarly here, helm is great for simple installation and templating, but the operator can do custom logic on the resources which helm simply was not designed for.
Please refer to the last section of this blog “Resources” where you can find more information about available tools.
There are a few choices if you would like to experiment with operators. I recommend going through kubebuilder’s quickstart.
If you decided to use this repository, there are a few prerequisites:
I’ve tested it with remote cluster and it works too, but doesn’t work with k3s/k3d.
The image will take a while to load as it pulls kubebuilder, Go binaries, and other components, so please patient
For me, the main takeaway from learning about Kubernetes operators was a much deeper understanding of the Kubernetes architecture, but most importantly the realization that the true power of Kubernetes is in its extensible API and universal control plane.
Whether it’s consuming operators or creating your own, starting with the mindset that Kubernetes is a platform to build platforms, is the path to levelling up your Kubernetes game.
Here are useful resources for further study.